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PREFACE 


The present book aims at investigating the logical founda- 
tions of the theory of Markov random processes, 


The theory of Markov processes has developed rapidly in 
recent years, The properties of the trajectories of such 
processes and their infinitesimal operators have been 
studied, and intimate conmnexions have been discovered 
between the behaviour of the trajectories and the properties 
of the differential equations corresponding to the process, 
These connexions are useful for studying differential equa- 
tions as well as Markov processes, The material thus accum- 
lated has made necessary a critical survey of the fundamen- 
tals of the theory, In particular, the usual statement of 
the Markov principle of "absence of after-effects" has been 
found to be inadequate and various authors have proposed 
different forms for a strengthened principle whereby a pro- 
cess is "strictly Markov." It has become obvious that the 
most natural subject for study is presented by Markov pro- 
cesses cut off at a random instant, All these and other 
ideas were originally introduced by different authors in 
different forms, according to the specific purposes of their 
specialized works - in which stationary Markov processes are 
considered almost exclusively, 


A general theory is built up in the present book which also 
covers non-stationary processes, Stationary processes are 
regarded as an important special case, Non-stationary pro- 
cesses are well known to te reducible to the stationary type 
by an artificial method requiring the passage to a more com- 
plicated phase space*), However, the stationary processes 
thus obtained are in a certain sense degeherate, so that 
this type of reduction is by no means always suitable, Then 
again, a concept of the Markov process that is more general 
in essence is closer to first principles than the concept of 
stationary Markov process, There is a canonical time scale 
for stationary Markov processes, In the general process 
there is no such scale and all the definitions have to be 


*See article 3 of Chapter 4. 
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invariant with respect to any monotonic continuous trans- 
formation of time. 


The theory cannot be adequately developed by extending the 
concept of a Markov process as a random function of a special 


type. 


For we are usually concerned, when studying Markov pro- 
cesses, not with a single probability measure but with a 
whole collection of such measures, corresponding to all the 
possible initial instants and all the possible initial 
states; in other words, we are concerned, not with one ran- 
dom function, but with a whole collection of such functions, 
with definite ihter-relationships, This is one of the rea- 
sons why the theory of Markov processes has to possess its 
familiar autonomy with respect to the general theory of pro- 
bability processes, The theory of Markov processes is built 
up in the present book without any reference whatever to the 
general theory of probability processes, 


This book cannot be used by the student to make his first 
acquaintance with the theory of Markov processes, Although 
we have not assumed formally any previous acquaintance with 
the theory of probability, in fact a reading of the book 
can only prove of value to someone already acquainted with 
an elementary exposition of the theory of Markov processes, 
such as is contained, for instance, in Feller's "Introduc- 
tion to probability theory and its applications," Vol, 1 
(Vwedenie v teoriyu veroyatnostei i ee prilozheniya), or 
Gnedenko's "Course of probability theory" (Kurs teorii 
veroyatnostei). 


The first introductory chapter contains a brief survey of 
the necessary concepts and theorems from measure theory. 
Any proofs that can be found in text-books are omitted here. 
The second chapter gives a general definition of Markov pro- 
cess and investigates the operations that make possible an 
inspection of the class of Markov processes corresponding to 
a given transition function. The more complicated operation 
of generating a subprocess is studied in Chapter 3. The con- 
nexion is revealed between the subprocesses of a Markov pro- 
cess and the multiplicative functionals of its trajectory. 
The most important classes of multiplicative functionals and 
subprocesses are investigated. Chapter 4 is concerned with 
the construction of Markov processes with given transition 
functions. The concept of strictly Markov process is 
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discussed in Chapter 5, Finally, Chapter 6 is devoted to a 
study of the conditions to be imposed on the transition func- 
tion so that among the Markov processes corresponding to 

this function, there should be at least one, all the traject- 
ories of which possess some type of continuity or bounded- 
ness, The supplement describes some of Choquet's results 
concerning the general theory of capacities, Measurability 
theorems for the instants of first departure are deduced from 
these results, A historical and bibliographical index will 
pe found at the end of the book, 


The present work is closely allied to a monograph now in 
the press entitled "Infinitesimal operators of Markov pro- 
cesses" (Infinitezimalnye operatory markovskikh protsessov), 
which is devoted to the task of classifying Markov processes. 
The two works should be regarded as the two parts of a single 
monograph on the theory of Markov processes, 


The present material comes from a series of papers and 
special courses given by the author at Moscow and Pekin 
universities. The author is grateful to his audience for a 
number of observations which he made use of during the final 
preparation of the manuscript, 


I must express my indebtedness and sincere gratitude to 
Mr, A.A, Yushkevich for his careful reading of the manuscript 


and various comments that made it possible to eliminate a 
number of inaccuracies and obscurities, 


E,B, Dynkin 
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CHAPTER 1 
INTRODUCTION 


1, Measurable Spaces and Measurable Sets 


1.1. Leto# be a system of subsets of a set 2 satisfying the 
following conditions: 


1,1.4,. If A€oM, then AG of *). 
_leledy. If A€oM(i = 1, 2, ...), then Arcot and 
NAc ot. 


We say in this case that the system of subsetso#is a 3- 


algebra in the space Q. Let C be any system of subsets of 
Q. The intersection of all the c~algebras in the space 2 
that contain @ is also a co~algebra, We refer to it as the 


oralgebra generated by @, and write it as o(@), 
If ois a c-algebra in the space yw and BCoM#, the aggre- 


gate of all sets A€o# that are contained in & forms a :- 
algebra in the space 8. We shall denote this c-algebra by 


of (S}. 

We shall call the system of subsets C of the space 2 a 
z-system if: 

1.1.3). It follows from A, 4,€ @ that AN A€ @ **). 

We shall call a system ¥ a \-system if it satisfies the 
following conditions: 


1,1.0,. 9€F. 


*7Z denotes the complement of A in Q2,1.e€,.2\A 
**The intersection of sets A and B will generally be written 
as ANB, 
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1.1.0,. If 4, 4€& and Ap 4,=6*), then A UAE SF. 


2° 
1,1.0;, If 4, A€fand 4,24,, then A,\A,EK 
1.1.0). If A, ....Ay--- EK and A, tA**) then ACF. 


It may be noticed that, if a system of setso#is similtan- 
eously a n~system and a }-system, it is a s-algebra, For 
1.1.4, follows from 1,1,C, and 1.1.0;. Moreover, it follows 


from the i uienatic Mak ie Cap ae properties 1.1.3), 
1.1.0, and 1.1.0, that, if A. BE of, then AUG Sate NAS) 

Ef, and consequently, if A;,A,,....4 n€ of then U A;€ 0M. Now 
let A, n€&M (n=1, 2,...). Then A= “Aro , and inasmuch as 
AtU An , we have G €o# by 1.1. o,. It follows from the 


relationship 





fo) ao 
4-4, 
1 1 


2 


Lemma 1,1, If the \-system # contains the x-system C 
then ¥ contains «(@), 


that \A,€od . Condition 1,1,4, is therefore satisfied. 
1 





Proof, The intersection ¥’ of all the i-systems contain- 
ing the x-system € is obviously a \-system, We shall show 
that this intersection is at the same time a t-system, The 
assertion of the lemma will follow from this, 


The class #1 of all the sets A such that An BC’ for all 
BE Cis readily seen to be a A-system, Since ¥,D@, we 
have ¥,5>¥/ This means that, if AC#’, BE@, then AN 
BEF’. 

We now put BE F,ifBnACF for all AC¥’. It may readily 
be seen that ¥> is a A-system, By what has been proved, 


*The symbol 6 denotes the empty set. 
**4,4A means that AAS AS...GAgS... and A= U4n . Simi- 


au 
larly, A,|A means that 4;24,2...24n2-.. and 


A=f\an 
1 
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FoDC, 30 that ¥,DF", This means that, if A. BEF’, 
then ANBE¥. Consequently ¥’ is a *-system. 


1.2. The pair (@, 4), consisting of a set 2 and avi- 
algebra of of subsets of 2 is called a measurable space. 


An important example of a measurable space is provided by 
the space (Ii, #7) , where H=|[s, 4] is a segment of the real 
line and #t is the c-algebra of subsets of this segment 
which is generated by all the intervals contained in . 
The values of s and ¢ can be infinite as well as finite, and 
we put J75=(— oo, +00), Ig =(— 00, t), Kao=[s, +00) + 


Let (&;, u£,) and (2,, -£.) be two measurable spaces, The 
mapping of the space 2, into @, is said to be measurable if 
the complete pre-image of any set from uf, belongs to vt. 
This definition is likewise applicable when the mapping is 
defined only on a subset $c. 


It may be noted that, if «o(@)=vf,, the only requirements 
that need to be satisfied by the mapping for the reflection 
ato be measurable are that the complete pre-image of any 
set from @ belongs to ef; and that: 2,= Uc: ; Where C,E C 


=1 
(this is proved simply by observing that the sets whose com- 
plete pre-image belongs to A, from a c-algebra in &), 


Obviously, if « is the measurable mapping of (@,, uf,) into 
(2,, wf) and g is the measurable mapping of (2, 4) into 
(25, #3) , then Be is the measurable mapping of (Q,, %,) into 
(Qs, x€;) , provided only that the domain of definition of 8 
contains «(Q,). 


The most important particular case of mappings is provided 
by numerical functions, i.e, mappings into the real line lta, 


Let xf be a s-algebra of subsets of 2. We shall say that 
the numerica: functions §(w)(w€Q) is measurable with respect 
to u or is -measurable, if the mapping defined by it of 
(2, v4) into (58. Bis) is measurable, i.e, if for any 

TE Bre ? 


{w:E@)ET) Eo. 


Since the intervals (f,-+00) generate the c-algebra #75 , 
the requirement for the function E(w) to be .£-measurable is 
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simply that, for any |, 
{w :&(w) > t} Ef. 


The concept of #~measurable function extends automatically 
to the case when the system of sets uf is a o-algebra in some 
subspace S of the space @ instead of in the whole of 2, 
It may easily be seen that in this case the domain of defini- 
tion of any uf -measurable function & coincides with 9, 


Let # be any system of real functions in 2, satisfying the 
condition 


1.2.4, If t€ Pana 


=| §(@) with §@)>0, 
WO=1 0 with E(w) <0, 
then n and¢ - 7 belong to Y, 


A system of numerical functions $ is called an .%-system if 
the following conditions are fulfilled: 


1.2,B,.1€ H. 


1.2.B,. A linear combination of any two functions of 3 


also belongs to #, 


1.2.B;, If §:€5@, 0<&,(w) tk(e) *) and&(w) is bounded or 
belongs to 2, then t€&#, 


Lemma 1,2, If an -/-system $ contains the character- 
istic**) functions of all the sets belonging to some 
n-systen @ , then # contains all the functions of & which 
are measurable with respect to o(@). 


Proof. The class of all sets whose characteristic functions 
belong to S forms the \-system F, Since FDU » we have 
F 2s(@) by lemma 1.1. 


*We write ante if a,saand a<acc...<a,<... Similarly, 
4n4@ means that an—>a and %>4:>..c>a,>.... 
**The characteristic function of a set A is the function 


_f{h ifoea, 
m={ 4, if ofA. 
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Let § be a non-negative function belonging to & and 
measurable with respect to o(@). We put 


twa f<teo< 434) 


on 


= =u 
eno. 2 kn 


Obviously ,0O<§&, t& and, in accordance with 1.2.3, 7 SCH. 

In view to 1.2.4 an arbitrary o(@)-measurable function 
n€.Lcan be represented as the difference between two o(@) 
-measurable non-negative functions of , The latter belong 
to H, as already pointed out. Consequently, we also have 


nt SH. 


1.8. Let uf; be a s-algebra of subsets of sets Q; ((=1, 2, 
... 2). We shall write 2,.x...x@, for the set of vectors 
(Wy, ..., @,) 9 Where o,€@; , andef, X... X uf, for the c-algebra 
nerated by subsets of the form A,X ... XA, , where A;€ 4; 
it may be noticed that the sets A,X ... XA, form a #-system), 


In the case when Q,=— ... =2,=—@, we shall write 2” instead 
of Qx... X@, , and similarly 4" instead of uf: X ... Kuta, 
when 4,= ... =%#,=d. 


Now let an infinite sequence of spaces 2; be given and a 
o-algebra of subsets ul; in each 2 space. We shall write 
2x ...xX2,xX... ror the space of sequences (0, w,..., 
Wass.) 9 Where w,€Q; , and ufjX... X4,X... for the o- 
algebra in this space generated by the subsets 


A,X 42 & «.. K An X Qasr X Lage X «-- (1.1) 
(n=1, 2,...; Ay € fj). 


When the factors are the same we shall write Q© and &~ 
for brevity. It may be noticed that the class of subsets of 
type (1.1) is a x-systenm, 


Lema 1,3. Let a, be the measurable mapping of (2,4) into 
(2; vi) (i can run either through the values 1, 2,...,2, or 
through all natural mumbers). Then the mapping o of the 
space (2, ) into (2,x2.xX..., i XuX...), defined by the 
expression 


a(o) = {a (w), a9 (0), ...}, 


is measurable, 
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Proof, Suppose for clarity that i runs through all natural 
numbers, The class of all the sets of ufjXu#2X ..- , whose 
complete pre-images with mapping « belong to 4, clearly 
forms a c-algebra. This c-algebra contains all the sets of 
type (1,1). It therefore contains Xu ..... 


Lemma 1,4. Let ut; be a c~-algebra of subsets of the space 
Q; @=1, 2) and let f(w,, ws) (w,€ Qi, EQ.) be an uf X of: -measura- 
ble function, Then for any fixed o,€&, f(v,, 2) is an A, - 
measurable function of o,. 


Proof, Let & denote the class of all functions in the space 
2,x2,. The system & of all functions f(w,, ,) for which 
our lemma holds is evidently an Y-system, This system con- 
tains the characteristic function of any set A,;XA2,. By 
lemma 1,2, it contains all functions measurable with respect 
to the c-algebra 4, X42. 


Lemma 1.5. For each ¢ of some set T let there be associa- 
ted a function x,(w) (o€ 2) with values in the measurable 
space (F, #). The necessary and sufficient condition for a 
function £(w) (w€ Q) to be measurable with respect to the o- 
algebra Wr generated by the sets (w: x,(w) ET} (ET. TE gis 
that . 


E(w) =f [x2,(), Xt, (W), 2.5 x4, (©), eee (1 2) 


where ¢ ty ....ty--- ET and f(xy, x ..., Xp, ...) is a PX- 
measurable function in the space E”,. 


Proof. The mapping of Q into the real line /}5—=(— oo, -++00) 
given by expression (1.2) can be written as the product fas 
where 


& (w) = {%2,(w), X4,(w), «6.1 He, (M) -- +f 


For an; ¢€T x,(w) defines a measurable mapping of (2, Wr) in- 
to (€, #).. By lemma 1.3, « is the measurable mapping of 
(2, of) in iS ae . By hypothesis, f defines the measurable 
mapping of (E*. @*) into (/7%, #73) . Consequently §=/a 
is the measurable mapping of (2. Nr) into (173. #72). Hence 
all the functions that can be written in the form (1.2) are 
W T -measurable, 


Now let & denote the class of all functions {(w) (w€Q), 
let H be the set of functions — that can be written in the 
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form (1.2), and let @ be the system of all w-sets of the 


forn 
{x,E Ti, sees x2 ET} 
(M1, oot fy woes ty OTs Bye eos PAE B): (1.3) 


The characteristic function of the set (1.3) is equal to 
Xr, [*2,@)] --- Xr, {*1,)}- 


Thus it can be written in the form (1,2) and belongs to &, 
Obviously, # is an -#-system, and C a x-system, and by © 
lemma 1,2, & contains all the functions §(w) measurable with 
respect to «(@)=WNr, 


2. Measures and Integrals 


1.4. Let (2, of) be a measurable space, The non-negative 
numerical function 9(A) (A€oM) is called a measure*)if, for 
any finite or denumerable collection of mutually disjoint 
sets A,, Ap --- of of, ?(U4,)=29(A,) . The measure satisfying 
the condition »(2)—1 is known as the probability measure. 


Let o be a measure given on the o-algebra of, Let f be an 
oM-measurable numerical function, given on some subset 2, 
of the space @, and let A be contained in @,, We shall 
say that the function f is ¢-summable in the set 4 if the 
finite limits exist, , 





+00 
lim Dee{orwea Zcfw< att} 
0 


nrc 


*All the statements and definitions contained in section i.4 
can be translated without difficulty to the case when the 
function 9(4) is allowed the value +o as well as finite 
values, (A particular example of such a function is the 
ordinary Lebesgue measure on an infinite numerical straight 
line.) We shall only encounter such functions in a few 
examples, however, so that, unless there is some special pro- 
viso to the contrary, the measure ? will always be understood 
to be a function that takes only finite values, 
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and 





lim d At oforwea Zefwe< er bm. 
A, 


NP 


The sum of these limits is called the (Lebesgue) integral 
of the function f on the set A with respect to the measure 9 
and is written as 


f f(w) 9 (dw). 
A 


If one limit is infinite but the other finite, the inte- 
gral is ascribed the value-too (tooif the first limit is 
infinite, and—ooif the second limit is infinite), 


We shall assume that the reader is familiar with the basic 
properties of Lebesgue integrals, The properties that we 
shall use most frequently are as follows: 


1.4.4. If0<f,(w)t f(o)for all w€A, then 


lim J f,(w) 9 (dw) = | f(w) 9 (de). 


1.4.B, If for all w€A f,(o) +f), |A|<g(w)and g is ¢- 
summable in A, then 


lim | fr (~) 9 (dw) = | f(w) 9 (dw). 
H poe (2.5) 


1.4.C, (Fubini's theorem.) Let of; be a o-algebra in Q; 
and 9 a measure in of, (i=1,2). Let f(w;, o) be an of, x of- 
-measurable function in 2, x 2, such that 


f [ fiflon 0) |e. cawo| 91 (dw,) < 00. 


2, 


f J fou 0) 9 (ae| 91 (do) = 
Q, Q 
=f | ffs «2 ¢:(don | ¢2@an)**), 


*"When a p-summable function is referred to without mention- 
ing on which set A, it will be a question of a function which 
is ¢-summable throughout its domain of definition. 

**If f is non-negative, the proviso regarding the absolute 
convergence of the integral on the left-hand side of (1.6) is 
unnecessary, 


9, 
“2 


(1.6) 
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The following property is a particular case of 1,4,A3 
1.4.4). If A,tA, then ¢(A,)t (A). 
1.5. We shall prove two useful lemmas, 


Lemma 1,6. Let x be the measurable mapping of (Q,, u4;) 
into .(Q,, uf.) and let © be a measure on 4,. Then the expres- 


sion 
oT) =¢ (2@)ET} TE A) (1.7) 


defines a measure in 42, We have for any .4,-measurable 
function f: 


f Fd 9 on) = f f[x(o)] ¢ du) (1.8) 
2 g, ° 


(more precisely, if one of these integrals exists the other 
exists and the two are equal). 


Proof, The first part of the lemma is obvious, 


Let & denote the class of all ¥-summable functions, The 
class of all functions for which equation (1.8) is satisfied 
is clearly an %-system, This system contains the character- 
istic function of any set of uf:, By lemma 1,2, it contains 
all the #,-measurable functions of 2%, Consequently, if the 
integral on the left-hand side of (1.8) exists for f, f 
satisfies (1.8). It can be shown similarly that (1.8) is 
satisfied for any function f for which the integral on the 
right-hand side of (1.8) exists. 


Lemma 1,7. let U, V, Z be three spaces and -4y, fy, hz 
o-algebras in these spaces; letF(u, z)(u€U,z€Z) be a real 
function measurable with respect to yXvulz ; and let 
P,(v€V) be measures in the s-algebra fz, the numerical 
function P,(T) being cfy-measurable for any Y¢€Az. If the 
integral : , 


G(u, v)= | F(a, z)P, (dz) 
f (1.9) 


converges for all w€U, v€Y it defines an vty X bv- 
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measurable function*), 


Proof, Let & denote the system of all vty X ufz -measurable 
functions F(a, 2) for which f F(a, z)P,(dz) converges for all 


Zz 
u€U, vEV . The system & of all the functions F(x, z) for 
which G(u, v) is uty X uly -measurable is evidently an @- 
system, It contains the characteristic function of any set 
Ay X Az(Av € uty, 4z € fz). Since these sets form a xr-system, # 
contains all uy X véz-measurable functions of by lemma 
1.2, 


3. Conditional Probabilities and 
Mathematical Expectations 


1.6. The triple (2, o4, P), where 2 is a set, of is ac- 
algebra of its subsets, and P is a probability measure in-of, 
is called a probability space, The points of 2 are called 
element events, the elements of o# are events, and the 
values of P(A) are their probabilities. Every o#-measurable 
function £(w) (w€Q) is called a random variable. The 
integral f E(w)P(dw) (if it has a meaning) is called the 


expectation of € and is denoted by M&., We shall also des- 
eribe as random variables o#-measurable functions § which are 
only defined in some subset & of 2 instead of throughout 
the space @ (it follows from the o4-measurability of § that 
Q.€o#), The expectation of such a § is given by the expression. 
Mi= f E(w) P (de). 
% 

Let SeQ and let p(w) and p(w) be two’ functions whose 
domains of definition contain §. We shall say that ¢=p 
almost certainly in © (in the sense of the measure P) and 
write os 

g=y @.c.&, P), 


if Pe g#)=0. Similar meanings attach to the expressions 


P<b @.c.8,P), gmo>e (.c.&, P) 
and so on, 


*In the case of a non-negative function F the requirement of 
convergence of.the integral is unnecessary, 
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Let % be a c-algebra in Qe, where GoM. Let the 
function §(w) be P-summable on the set 2, Every .{-measur- 
able function satisfying the relationship 


E(w) P (dw) = f MG]4) P du). 
: J (1,10) 


for any A€v£ is known as a conditional expectation of .— with 
respect to 4 (written as Mé(€].4)). 


Such a function may be shown always to exist, I+ may easily 
be seen to be defined only up to an arbitrary set of the o- 
algebra £ having P-measure zero, Since it is impossible 
to conclude from the equations M ¢|t)=9,M(E|4)=4 that 
¢(o)=$() for all w, the only conclusion that may be drawn 
being that p= (a.c, 9, P ), we shall avoid paradoxes by 
writing M(€|4)=¢ (a.c. &,p) instead of MG|.£)=o9. 


It should be noticed that the function M(]-4) is unaffec- 
ted by the values of the function § outside §, 


If B€oM, the characteristic function Xp(w) is an of- 
measurable function, The function M(x,|£) is called the 
eonditional probability of the event B with respect to the 
c-algebra # and is denoted by P(8|ué). This function can 
also be characterized as the uf-measurable function satisfy- 
ing for any A€cé: 


P(AB) = { P (BJA) P (du). 
J 4! (1.11) 


(It is defined only as far as an w-~set of # having P- 
Measure zero,) 


We shall mention a few examples*), 


1.6.1. Let be an algebra consisting of two elements: 
the empty set 9 and the whole of the space @, Then con- 
stants are the only vf) -measurable functions and we have 
from relationship (1.10) with 4=@. 


M (| fo) = ME 
*The o-algebra .f does not contain non-empty sets of measure 


zero in examples 1.6.1 and 1.6.2, so that the function P (é| et) 
is uniquely defined for any §,. 
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1.6.2, LetAG€o# and let O<P(A)<1. The four sets 


{0, A, A, Q} form a c-algebra 4. We have 


( Fuy | Eo) P@e) for w€A, 





MELO={ | * _ 
Pa! § () P (dw) for o€A, 
( P(BA) 

eo ee 
aE) A w€ A. 





These expressions establish the connexion between the 
general concepts of conditional mathematical expectation and 
conditional probability with respect to a given o-algebra 
and the popular elementary concepts of mathematical expecta- 


tion and probability that some event A might occur, 


We shall deduce the fundamental properties of conditional 
mathematical expectations and conditional probabilities, In 
the statement of these properties, .4 denotes a c-algebra in 


Seg , it being assumed that {So#, 


1.6.4, If § is £-measurable and P-summable in ©, then 


ME[f)=E (a.c. @, P). 
1 6.4. For any constant c, . 

M(c[#)=c (a.c.&, P). 
1,6.B. If & and 7 are P-summable in @ and 


t>7n (a.c.G, P), 


then 


MO|OQ>M(i[4) (a,c. 8, P). 


1,6.B,. For every AC oM 


O<P(AlA <1  (a.c.@, P). 


(1,12) 


(1.13) 


(1.14) 
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If p(@\A8)=0, then 
P(AJ-#)=1 (a0, P). (1.15) 
1.6.0. If.§: and & are p-summable in @, c, and c. being 
arbitrary constants, then 
ME +o | A) =oME[OQ+oMG|  @.c., P). 
1.6.D, If § is P-summable in © and 
O<eté (a,c, 8, P), 
then 
ME, [€) t ME} 4) (a.c,9, P). 
1,6.D,. If 4,fA, then 
P(Anft) tP(Afé = (a.¢. 8, P). 
1.6.8, If » is P-summable in @ and 
xl<meees [Gola ---5 Gt (a.e.8, Py, 
then 
MG, [€)>M(E]4)  (@.0.2, P). 


1.6.F. If & and 7 are p-summable in © and { is 4- 
measurable, then 


MGn[vf)=§M(nf[£) (a... P). 


1.6.G, Let 259,598,, let vf, be a c-algebra in @, and 
4 a spalgebra in &, , where 4,904, of, 20M and 4,9, € 4, 
for all 4,€u#,*) . If the functions f, and 7 are P-summable 
in 8, and the function § is equal to zero in 9,\ 9, and 
induces anif,-measurable function in @,, we have 


MGn|4)=MEM(q}t) Joh] (@-c.8,. P) **). (1.26) 
If A€ut, BEoA, then 


P(AB) A) =M{x,P(Bl4)f4} (2.0.8: P). (1.17) 


*That is, ,(%)|Cut,3 in the case when 0,..0,, this require- 
ment reduces to the simpler condition 4,¢C.%.. 

**The first factor in the product §&M(y]%.) vanishes whilst 
the second is not defined in 9,\8 . The product is 


reckoned to be defined and equal to zero in Q\oG, 
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1.6.H, If the functions & and 71 are P-summable in © and 
the function § is vf-measurable and equal to zero (or not 
defined) outside 9 , then 


ME = M [EM (| vf). 


1.6.1. Let be a c-algebra in the space 2, If yis P- 
summable in @, then 


M7 = M[M(n[-4)). (1.18) 
If A€o#, then 
P(A) =M|P(A]-4)]. (1.19) 
We now turn to the proof of properties 1.6.A-1.6.I. 
Property 1.6.4 is an obvious consequence of the definition 
of conditional mathematical expectation, On putting E=c, 


in 1,6.A, we get 1.6.4). 


Proof of 1.6.B.-1.6.B 
any A€vét: 


1: We have by (1.10) and (1.12), for 





ME] 4) P(dw) = { EP (dw) > | 4P (dw) = 
) s) >fr mn (1.20) 


4 
=f M(n]et)P @u). 
We put : 
A,={o:M Gl) <MO]o—s}. 
Obviously 4,€ 4 and 


Jf me1or de) < f Mot P@uy—1 P(A. (1.21) 
4, A, 
It follows from a comparison of (1.20) and (1.21) that 
P(4,)=0. But 
{ME} €) <M (q]@} CU An 
whence (1.13) follows. 


To deduce (1.14), we only need to notice that 0<y,<1, 
and to make use of properties 1.6.B and 1.6.4, already proved, 
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Finally, if P@\48)=0, then x,>1 (a,c, Bs? and, by 
(1.13) and 1.6.4, P(A| 4) =M(x,|t)>1 (acc, §,P), Taken 
in conjunction with (1.14), this gives expression (1,15). 


Proof of 1,6,C. In view of (1.10), we have for any AE Bb: 
S&P Go) = f ME&IOP Cu); 
4 4 


S&P do) = f MG&IOP je), 
A A 


so that 
Jf Cabs coke) P du) = f tM G,]o) +-oM GL O1P (du); 
A 


A 
which is equivalent to proposition 1,6.C, 


Proof of 1.6.D-1.6.D,. ‘In accordance with 1.6.B, it 
follows from the relationship 


0<h< ... KES... = (a.c. &, P), (1,22) 
that 
O<MEIAK -.. <MGEJA<... (2.0.2,P). (1,23) 
By (1.10), for any AC 4, 
f MGnle)P Gu) = f EP do). (1.24) 
By 1.4.4, it beitces from (1.22), (1.23) and (1,24) that 
if lim M &n[x€) P (dw) = ii lim §,P (dw) 
4 


and therefore 
MCn|et)=MEGft) (a.c. &, P). 


We can prove 1.6.D, simply by applying 1.6.D to the 
quantities %4,. 


Proof of 1,6,E, We put 





Gt = sup baths Oh a= inft ke 
k>0 k>o uae 


Obviously, 
O<n—Gtn—t & O<ntGtntt (a.c. 8, py. 
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Therefore, by 1.6.D, 


M(n—Gr fet) [M(n—E]t) & M(ntta fA) PM +eyA) 


a.c, &, P). 
Thus ' : 


M(i]O) | MEf® & Mf) [MEI (8.¢.8, Pp). 
It remains to remark that, by 1.6.B, 
M(G [A <MG,[O <M]. 


Proof of 1.6,F. We fix any function 7. P-summable in © 
and put §€.%, if '; is P-summable in @, By 1.6.C and 
1.6,D, the class S¢ of all €for which 1,6.F is satisfied 
forms an L-systen in ©, It follows readily from (1.10) 
that S#? contains the characteristic functions of all the 
sets of £, By lemma 1.2, $ contains all the vf-measurable 
functions of , 


Proof of 1.6,G, We have by (1.10) and 1.6.F for every 
AE hy: 





f M {EM (nf of2) 141) P (do) = a EM (n]f:) P (da) = 


= [malar a= J Mal) P (do) = 
Ag, AQ, (1,25) 


= f EP (dw) = A EnP (dw). 


Ag, 


Since the function M{tM(nJv4.)Jot:} is uf,-measurable, 
ee ae follows from (1.25). On putting t=, 4 NXg» We get 
1.17 


We prove property 1.6.H simply by applying 1. ( .G to the 
case ,=2, A=, 3,=—8, 4,=.. We get (1.18) on 
putting §—9, t=1 in 1.6.H, and (1.19) by putting 1=yx,- 
in (1,18). 


4. Topological Measurable Spaces 


1.7, A topological space is a pair (£, @), where F isa 
set and € a system of subsets of E& such that: 
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1.7.4). The empty set and the whole of space E belong to @, 


1.7.4,. The intersection of a finite number, and the sum 
of any number, of subsets of C again belongs to @, 


The elements of the system € are called open sets of the 
space E, Every open set containing a point x is called a 


neighbourhood of x, 


We describe AGE as closed if AEC, The intersection 
of all the closed sets containing a set Bis called the 
closure of 8, The closure of Bis the least closed set con- 
taining B. id 


Let x, be a sequence of points of the space (F, @), If 
every neighbourhood of the point * contains all the elements 
of the sequence *, apart from a finite number, *, is said to 
be convergent to x and we write %,—7%* , 


The topological space (F, @) is said to be compact if a 
convergent subsequence can be chosen from every sequence. It 
may easily be shown that a finite covering can be chosen from 
every denumerable covering of a compact space E by open sets*), 
If this condition is satisfied for any (not merely denumerable ) 
covering of E by open sets, (FE, @) is said to be a bicompact 
topological space, 


We refer to (FE. @) as a Hausdorff topological space if, for 
any two distinct points x, y€E disjoint sets U, VE @,can be 
found such that x€U, y€V. A bicompact Hausdorff space is 
called a bicompact. 


A subset A of a topological space (E, @) is said to be 
everywhere dense in F if, for every non-empty CE€@ , the 
intersection ANC is non-empty, If a denumerable everywhere 
dense set exists in E, (E, @) is described as a separable 
space, 


Let E’ be an arbitrary subset of E and let @’ be the class 
of all subsets £’ expressible as intersections F’NC , where 
C€@. Evidently, @’ satisfies conditions 1.7.4)-1.7 Aas so 
that (E’, @’) is a topological space, Every subset of a topolo- 
gical space can thus be regarded as in turn a topological 


*)A system of sets @ is said to form a covering of £ if each 


point x¢€£ belongs to at least one set of @, 
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space, In particular, it is meaningful to speak of the com- 
pactness or bicompactness of a subset £’, 


If E can be expressed as the sum of a denumerable number of 
compact (bicompact) subsets, the space (E, @)is said to be 
o-compact (g-bicompact), If there is an open set U and a 


compact (bicompact) set K for every point such that x€UCK, 
E is said to be locally compact (locally bicompact). 


Let (E;, @,) and (E, @,) be two topological spaces, Let @:X 
C2 be the class of subsets of the set £,xXE, expressible as 
sums of "rectangles" C,XC.(C,\E C1, C,E @,) ~ The topolo- 
gical space (E, XE, @1X G2) is said to be the topological 
product of spaces (F,, @,) and (E,, @,). 


It should be noticed that (%n yn) —>(*, y) in(—E, xX FE, @1X @,) 
when and only when x,>%* in(E,, @,jand yn>y in (Ey @2). 


A metric space is defined as a pair (£, p), where E isa 
set and p a non-negative function in EXE, satisfying the 
following conditions: 


1.7 .B, op Y=ey, ¥) 
LateBoal Y+e 2) Spex, 2) 


1.7.3 p(*%, y)==0 when and only when x=y, 


3° 
The set of all points » satisfying the condition e(™ y)<e 
is called an e-neighbourhood of the points x and is written 
U(x). A set U is said to be open if, for every point x€U, 
there is an e>0 such that U,(x)&2U. The class @ of all 
open sets U satisfies conditions 1.7.4,-1.7 Ay. Every metric 


space can therefore be regarded as a topological space and 
all the concepts introduced for topological spaces can be 
applied to it, Topological spaces which can be got with the 
aid of the above construction from some metric space are 
described as metrizable. 


A sequence of points +, of a metric space (E, p) converges 
to a point x when and only when o(x,, x) +0. 


A metric space is said to be complete if every sequence x, 


satisfying the condition lim p(x,,, x,)==0is convergent, 
Mm, APO 


The simplest example of a metric space is the set of all 
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real numbers R with distance p(x, y)=|y—x]. This space is 
complete, separable, o-bicompact and locally bicompact, 


1.8. Let (E,, @,) and (EF, @2) be two topological spaces, The 
mapping « of £, into FE, is said to be continuous if the com 
plete pre-image of any set CE @, belongs to @,, If a is the 
continuous mapping of (Fi. @;) into (EF, @,)and B is the contin- 
uous mapping of (Ez G2) into (Ey @;3), Ba is the continuous mapp- 
ing of (E; @1) into (Ey, @s). 


If a is the continuous mapping of (Ei, G1) in (Fa @2), it 
follows from *x,—~x that a(x,)—a(x). The question arises 
as to whether the fact that a(x,)—a(x) aS *,->* implies the 
continuity of the mapping «. Although this is not in general 
true, it is true in the case of metrizable spaces, 


It follows from 1.7.B, that, if *,>*x, y,—>y, then p (ar ¥n)> 
p(x, vy). The function” p(x, y) is therefore continuous in 
the topological space (EXE. @ X@) (@ being the system of 
all open sets of (E, p))*). 


Let T be a subset of the metric space (E,p), We write 
p(x, T) for the lower bound of p(x, y) over all yer. It 
follows from 1,7,B,-1.7.B, that 


Ip Ty—pvy. I] <p(x, »), 


so that p(x, T) is a continuous function of x, 


1.9. A topological measurable space is a triple (EF. C, #), 
where the pair (F, @) is a topological space and the pair (£, #) 


is a measurable space, 


The simplest and most important particular case of a topolo- 
gical measurable space is provided by spaces of the form(E, @, 
o(@)). We shall sometimes write these as (E. @) for brevity. 
The elements of the c-algebra o(@) are known as Borel sets of 
the topological space (E, @). 


Let (EF, @1 #1) and (Ex C2 #2) be two topological measura- 
ble spaces and a the mapping of F, into &,. It is meaning- 
ful to speak of a as being measurable or continuous, We shall 
show that the continuity of « implies its measurability if 
#:=2(@1) and #,=0(@,). For let #& denote the class of all 
subsets of E, whose complete pre-images belong to o(@,). 


*)The space (EXE, @x@) is obviously metrizable, 
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Obviously, ¥ is a X\-system and contains the x-system @2, 
By lemma 1.1, ¥20(@.). 


There is no point in confining ourselves to topological 
measurable spaces for which @=«(@), since this considerably 
restricts the possible applications of the theory. Various 
assumptions will be made regarding the connexion between # 
and C , depending on the circumstances, The most important 
of these assumptions are stated below, 


1.9.4. # is a c-algebra generated by some sub-system of 
the system @, or what amounts to the same thing, #=«(#N@), 


1.9.B. For every UEC #NE there is a #-measurable con- 
tinuous function f(x) such that f(x)#0 when and only when 
xE€U, 


Lemma 1,8. Let the topological measurable space (EF. @. #) 
satisfy conditions 1,9,A-1.9.B. If the 2-system { contains 
all the bounded continuous measurable functions in E£, it 
also contains all the measurable functions belonging to #. 


Proof, The system of sets #N@ is a x-system, where, by 
1.9.4, «((#N@C)=#. In accordance with lemma 1,2, we only 
need to show that the characteristic function of any set 
VE FNC  delongs to 3%. We take the function f(x) defined 
by condition 1,9.B and put 


1 
f= lf L 
n 


Yn (2) = 


1, af [a[> 
| 2I4], if [ul< 


The functions f, are continuous, bounded and measurable; 
they thus belong to #. Since O<f,fx%q » We have XvE KH. 


lemma 1,9, let %----, @,... be measurable mappings of a 
measurable space (2, o#) into a topological measurable space 
(E. @. #), satisfying conditions 1,9,A-1,9.B, If a,(w)>a(w) 
for any wé€Q2, the mapping a is also measurable, 


Proof, Let UE CN#H and let f be the function defined in 
condition 1.9.B,. We have 


wm 


MEY=Y U A {lfteon|> zh. 


mal kel n=m 
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The function f[a,(w)] is measurable, so that ({|/[2,()]|> 
><> €o#. Consequently (a(w) EU} E of « It may easily be de- 
duéed from this, by 1.9.4, that {a(w)E€T}Co# for every TEM. 


A term may be introduced that we shall occasionally find 
useful below, 


We shall say that the points ¢: define a canonical sequence 
of subdivisions {a?} of the interval A if: 


1) for any r,4=[Jaga? and a? being disjoint for i4/; 
k 


2) the points “2 is the right-hand end of the interval 
Ay 3 all the points #2 belong to A3 


3) n 


the interval %, 


: n 
lim sup | Az| = 0 where |Ai|=|¢—f#_,| is the length of 


Lemma 1,10. Let A be a numerical interval, (2. o#) a measur- 
able space, and (FE, @, #) a topological measurable space 
satisfying conditions 1,.9.A-1.9.B. Let the mapping F of the 
product AX @ into E satisfy the conditions: 


a) for any f#€A, the mapping fF is a the measurable mapp- 
ing of (2, of) into (E, @); 


b) for any w€2, the mapping F is the mapping, contin- 
uous from the right, of a into (E, @)*). 

Then F is the measurable mapping of (A x 2, #, x of) into 
(E, #)- 

Proof. Let the points “ define a canonical sequence of 
subdivisions {A?} of the interval A. In view of a), the mapp- 
ings F, of the space AX@2 into E are given by 

Fr(t, w) = F (ff, ©), if tE Az, 


and are measurable, In view of b), 
F,(t, 0) > F(é, ©) 


as n->co and F is measurable by lemma 1,9. 


A metric measurable space is the triple (F.p, @) , where 


*)That is, for any €9 and H¢A Fi,«)>F, 0), If t}to . 
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the pair (E. p) is a metric space and the pair (£. #) a measur- 
able space, When the c-algebra # is generated by the system 
of all open sets of the space (E.p) we shall write (£.p) for 
brevity instead of (E,p,9). It may be noticed that condi- 
tions 1,9.4 and 1.9.B are satisfied in this case: p(x, U). 
can be taken as the function f(x), 


5. The Construction of Probability Measures 


1.10 The following general theorem is usually taken as the 
basis for constructing probability measures, 


Theorem 1,1. Let C be a system of subsets of a set E 
satisfying the conditions: 


1.10.A,. FEC. 


1,10.4,. If AEC, then A can be expressed as the sum of 
a finite number of disjoint elements of CC. 


1,10.4,. If A, BE@, then sBee. 
Let ®(4) be a function in system @€ such that: 
1.10.3). ®(A)>0 sk every ACC. 


1,10.B,, If A=UA4s, where 4, 4, A, .-- €@ and 4,4,=6 
for jzj , then * 


9(4)= ¥ oy, 
1.10.B;. @(E)=1. 


Then there exists a unique probability measure P(A) on the 
s-algebra o(@) such that 


P(A)= (A) (AE @). (1.26) 


The uniqueness of the measure P follows directly from 
lemma 1.1. For let P be a measure in o(@) which, like P, 
satisfies condition (1.26), We put BEF if $B (s)—P(s) . 
Obviously ¥ 2 @ and is a j-system, Since @ is a x-systen, 
F 2s(@) by lemma 1,1, 


In order to construct at least one measure P satisfying 
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condition (1.26) we put for each BSE 
P(B)=int > P(A,). 
n=l 


where A,,..., Ay, .--EC and the lower bound runs over all the 


systems {An} such that U A,2B We consider the system # 


of all sets B for which” P(8)--P(B) = 1. It may be shown 
that the system # contains @ and is a c-algebra and that 
the function P(8), if it is considered only in the s-algebra 
#-, is a measure satisfying conditions (1.26), This is all 
the more true if P is considered only in the s-algebra «(@), 
which is contained in @. 


1.11 We take an important example, Let E=[a, 6]*) and let 
the function F(u) (a€[a, 5]) satisfy the following conditions: 


1.11.A.F(uz) is non-decreasing; 
1.11.B. F(u) is continuous from the right; 
1.11.C.F()=1 


We write @ for the aggregate of all intervals (s, ?¢] 
(@<s<t<b) and [a,t] @<t<o) and put 


®(s, 1= F@)—F(s), Ofa, 1=F (H. 


It may easily be seen that @ satisfies conditions 1,10.A,- 
1.10.45, and © the conditions 1,10,B)-1.10.B;. The pro- 


bability measure P(A), the existence of which is asserted 
in theorem 1.1, obviously satisfies the condition 


Pla, ]=F (2). Gey) 


1.12. Theorem 1,2. Let T be an arbitrary set, 7 a subset 
of T, (E, @, #) a s-compact topological space satisfying 
condition 1.9.4. We write ¢? for the class of all functions 
?() on the set 7 with values from £ and W% for the o- 
algebra in the space E” generated by the sets {9:9(‘)€T} 
(¢€7,TE#) . Let the function®,,....4 (1 -... Tn) Ty ..., hE #) 
be given for everyn=1,2,... wand any 4,...,4,€7 , and let 
this function be subject to the following requirements: 


*)a and 6 can take either finite or infinite values. 
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1.22.8. O.000g Gy T= O14, P+ Pn) for any 
1 n 
permitation (i,,.... i,) of the numbers 1, 2,....2.. 


1.12.B, If the values of all the arguments except one of 
the function ©:,...,1,(f ---» Ts) are fixed in an arbitrary 
manner, the function is a measure with respect to the remain- 
ing argument. 


1,12.C. O(AE)=1 5 


Oh ty tC oe Pras EV= 91,0, (Pv ++ Trt) 


(n> 1). 
Then there exists the unique probability measure P in the 
o-algebra dy such that, for anyt,,..., 4,67, Ty... r,€ #? 
P(oG)ETL .... PGrETn} =O, ..,, Pix dee Dale (1.28) 


We prove this theorem by observing that the subsets of the 
space £7 that have the form 


A=(e: 9 Gel ---, eG) ETa} 
(a= 1, 2,...; ty... GETS Ty... TCX) 


form a system @ that satisfies conditions 1,10.4)-1.10.4;, 
It may further be verified that the function ®, defined in 
the system € by the expression 


DBloAETy, ...5 9n)ETr} = %,..., 1,71 eve, Ty). 


satisfies requirements 1.10 -B,-1.10.B, (the assumptions 


regarding the topology of the space (E. #), introduced in the 
statement of the theorem, are employed in verifying property 
1,10.B,). After this, theorem 1,2 follows from theorem 1.1, 


CHAPTER 2 
MARKOV PROCESSES 
1, The Definition of Markov Process 


2.1 The clearest picture of a Markov process may be got 
from the following description, Suppose a particle is in 
random motion in a space E during the interval of time [0, %, 
If the position of the particle is know at the instant t, 
supplementary information regarding the phenomena observed 
up till the instant ¢ (and in particular, regarding the 
nature of the motion until ¢) has no effect on prognosis of 
the motion after the instant ¢ (for a known "present", the 
"future" and the "past" are independent of each other), The 
instant € at which the motion is cut off may be random, 


The accurate definition is as follows, Given: 


a) a function Cv) in a space @, taking non-negative 
values (including the value oo ‘; 


bd) a function «x(t, o)—-x,(w), defined for w€2, t€ [0, fw) 
and taking values from a measurable space (E. #)(the c-algebra 
# is assumed to contain all the subsets consisting of a 
single point); 


ce) for each 0<¢s<t.a c-algebra of: in the space Q,— 
{w :C(@) > ft} 3 


ad) for each s>0, x€E a function P,.(4) on a s-algebra 
oM* in the space @ containing of% for all t>s, 


We shall say that these elements define a Markov process. 
X=(x, 6 oft, P, ,) if the following conditions are satis- 
fieds 

2.1.A.s<t<uand A€ oH, then {4,C>u}€ oft. 

2.1.8, (ETIEM sor any 0<s<t. PE #*), 

e find, in particular, on setting r=e£ {>f} CoA] 


for any 0<s<t. 
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2.1.C, P,,, is a probability measure on the c-algebra , 
2.1.D. For any 0¢s<t4,T€ 9 
P(s, x; t, TY= Ps, 2% €T) (2.1) 
is a @-measurable function of x, 
2.1.E. P(s, x; s, EX x)=0. 
2..F, If 0<s<t<u, «€£, TE, then 
Psa {xu€ToMi}=P(t, x5 4T) (@.c.21 Paz). (2.2) 


The set @ is called the space of elementary events. The 
measurable space (E, #) is called the phase space, the quantity 
% the instant of cut-off (or the life), the function P(s, x;4,F 
the transition function of the process xX, With w fixed the 


function x,(w) @€[0,¢(w))) defines the trajectory of the pro- 
cess in the space £ corresponding to the elementary event o, 


The c-algebra of: can be pictured as the class of events 
which are observed during the interval of time [s, ¢#]. The 
value of P, (A) (A€o#*) is interpreted as the probability of 
event A on condition that the particle is at the point x at 
the instant s, 


Condition 2,1.F can be replaced by the following: 
2.1.F°, If0csct<u, x€E, AC o#i, then 


Py, 2 (A. ty€ T= fre. xg a, T)P,, 2 (do). (2.3) 
A 


For by definition of conditional probability (see sec, 1.6), 
2.1.F is equivalent to the combination of 2.1.F* and the 
requirement that P(t, x,; 2, T) be an oMt -measurable function 
of w, But this latter requirement follows from condition 
2.1.D. In fact, the mapping of 2, into the segment 2=[0, 1] 
defined by the function P(t, x,; u, T) is the product of the 
measurable mapping of (Q;, of#{)into(E, #) defined by the func- 
tion x;(w) and the measurable mapping of (£,#)in (7, #%) 
defined by the function P(t, x; 4, T). 


An important class of Markov processes is provided by those 
for which ((w)=-++oo for all w€2. We shall refer to these 
as non-cut-off processes and write them as (x;, cM, Ps,z). 
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Remark, The above definition of Markov processes can be 
somewhat widened, We fix a set T of real nuzbers and assume 
that: 


a) C(w) can take only values from T and the improper value 


+ co $ 


b) the function +,;(w) is defined for values of t€T less 
than (C(w) 3 


c) the c-algebras of} and off’ and the functions P,,, are 
defined only for s, t€7T. 


If this assemblage satisfies conditions 2,1,A-2,1,.F, it is 
said to define a Markov process in the time set T*). On 
putting T=[0, +oo), we return to the fundamental case con- 
sidered at the beginning of sec. 2.1, If 7 coincides with 
the set of all non-negative integers (or positive integers), 
we speak of a discrete Markov process or Markov chain, We 
shall save ourselves unnecessary complexity in the future by 
confining ourselves to the fundamental case T=[0, +09), 
since the case of an arbitrary set T does not differ from 
this in any essentials, 


Some further concepts related to Markov processes may use- 
fully be introduced, 


We shall describe as a Markov random function in the phase 
space (E, #) and in the time interval /J=[a, 6] the aggregate 
of the following objects: 


a) the function ((w) in some set @ with values from the 
segment [a, b]$ 


b) the function x,(w)=x(t, w) defined for »€2,#€fa, C(w)) 
and taking values from £3 


c) for each #€/ the c-algebra of, in the space 9,= {£>12}; 


a) the probability measure P on the o-algebra of in space 
Q, containing &#, for all t¢/. 


It is required here that: 


*)The concept obviously remains meaningful in the case of 
any ordered, or even partially ordered, set 7, 
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2.1. a. (4, ET} Co#, for any TE, tEl. 
2.1, a. of, [2,1 Go, for any t<u€/:. 


2,1. Qs. P{x,€P]o4,} =P {x, CT] x,} (a.c, 2, P)*) for any 
tues and TEM, 


We shall speak of a Markov family of random functions if 
there is associated with each pair s>0, x€E a Markov random 


function x* 7 = {x?'7, (7, cf? 7, P* 7}in the phase space (E. #) 
and in the time interval [s, oo). The following conditions 
mist be fulfilled here: 


2.1. 8, P(s. 5 4.1) =P" (x? "ET} is a @-measurable function 
of x, 


2.1. Pe Pls, x; s, EX x)=0. 


2.1. fy. PY 7fh EL |x? "=P x? 2D(acc, OF? p*?), for any 
Oc¢s<cit<cu TE. 


Let X=(xp 6, offi, Ps, 2) be an arbitrary Markov process. We 
fix ans>0 and x€E and put 
C& 2(w) = max (2 (w), $), 
XP? (w) = x, (0) (Els, co), EQ), 
My? =o; (Els, 00)), 
P*7(A)=Ps,2(A) (AE of’). 


The elements (C°*, x#*, of”, P**) are easily seen to define 
@ Markov rendom function in the time interval [s, co), ‘The 
aggregate of these functions for all s>0,*€E is clearly 
a Markov family. 


On the other hand, a Markov process may readily be associa- 
ted with each Markov family of random functions X*” — (x? %, 
c*?, ot? 7, P**), For let G be the class of triples (s, x, w), 
where s>0, x€E,.w€Qs2, and let 


G(s, x, 0) = 0% 20), 
%,(S, X, w) = x7 (0), if £E[s, C% 2(w)) 


*) P{—|x,} denotes the conditional probability with respect 


to the c-algebra in the space &, generated by the sets 
{x€ BY (BEB) , 
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fwe define the function <x,(s, x, ©) in an arbitrary manner for 
t€[0, s)]. For each 4c G0 we put 


A® == fw: (s, x, w)€ A}. 


lie write of for the class of all sets AcS& , such that 
A®?€ of * for any x€E, and of: for the class of all Ag@ 
such that A” ”€ of? * for any x€E, It can easily be seen 
that of? is a s-algebra in the space 0,— (f>1} and of a c- 
algebra in the space 2, where ,7>.#2 for all t>s. 
Finally, for every A€o#* we put 


P,, 2 (A) = P* 2(A® 2). 


_We leave it to the reader to verify that the system (pF oA, 
P, 2) satisfies conditions 2.1,A-2,1,F and is therefore a 
Markov process, 


2.2. We shall use the notation } for the c-algebra in the 
space Q, generated by the sets (' x, (ET) 2, wen, TE B)s 
and W* for the c-algebra in the space generated by the sets 
{o: x,@)ET} WER TER). 


In view of conditions 2,1,A and 2.1.3, WN? Soff, and df" Gof* 
Obviously, (* © Wt, P, .)is a Markov proces3 as well as 
(x, 0 off, Py ix)® 


Furthermore, we put A€o#* if there exist sets B, and 8, 
a for every x€E such that 3,2 AGB, and P, ,(B,)=P, 
awl\2) . ; 


On putting Ps, 2 (A) = Pa, (Bi) = Py, 2(B2), we extend the pro- 
bability measures P, . to the c-algebra of®, This evidently 
does not affect the fact of (x,, 6 of, P,,2) being a Markov pro- 
cess; it is not affected even by replacing the omalgebras 
oM: by the wider o-algebras off! » Which are constructed as 
follows: we putAGo#if AE of’ ,AGQ; and for every x€E 
there exists 3p Eo#; Such that 


P,,2(A \ AB) = Py, 2(B \ AB) =0. 


Theorem 2,1, Let (% [ of: P,.) be a Markov process and 
let O0<¢sct . Tf BEN, then 


P,, (BoM) =P, (B) (2.04%, Py.) (Bed) 


*)It is clear from (2.4) that, for any Bed! the function 


P,,y(B) can be constructed in accordance with Po.2 as far as 
a yeset ¥ , for which P(, 45, I’)=0. 
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If t is W’ measurable and P, .-Summable, then 
Ma, a2 & JoMe) = My, ot (a.c, Q, P,, a) (2,5) 
Proof, Expressions (2.4) and (2.5) will be proved simultan- 
eously, We start by showing that (2,5) is satisfied for 
t=f(x.)5 where u>t and f is a bounded #@-measurable func- 
tion in space &. Let - be the class of all bounded func~ 
tions f(x) (x€E) and § the class of all #-measurable 
functions f such that f(*,) satisfies (2.5). Clearly, # is 
an .f-system, In view of 2,1.F, d contains the characteris- 
tic functions of all sets Té@. By lemma 1.2, dS? contains 
all bounded #-measurable functions, 
We now prove that (2.4) is satisfied for every set 
B= (xu, ET ...%u,€Ta} Cap -- Os Ty... Tae #)- (2.6) 


If n=1, this is true in view of 2.1.F. We now use induc- 
tion. Let 


By= {xu,E0i}; Be (Xu ETe -- + Xun ET). 
Obviously, B=B,B, » and by 1.6.3, 
Pe, 2(BLoM)=My, o{Xp,Ps, 21BsloMilloM} (240-2, Py, 9) 
We have by induction: 


Ps, z Bo} Ms.) ae Pu,, ey, (B:) (a acy Quy Ps x). 
Therefore 
Ps, 2 (Bh of} = M,, 2 {15 Pu, Dy, (Be) J oh} = 
a Mg, 2 {fF (%u,)] oft} (a,c, 2, P., a) 
where 


FO) =p, (XP, (Br). 


2 applying the particular case of (2.5) already proved, we 
ve 


Me, x {f (%u,) [o%3} = Mi, a, F (Xu) (a.c, Q, Ps, a) (2.7) 


On the other hand, by 1,.6.H and our inductive assumption, 


MARKOY PROCESSES 31 


P,, y(B) = M;, y (Zp, y(Bofo%s,}} = 
— M,, y BP cy, (B2)] = M,, yf (*,,). (2 8) 


On comparing (2.7) and (2.8), we conclude that (2.4) is 
satisfied for all sets of the type (2.6). 


We now write .£ for the class of all Ps,z-summable functions 
E(w) (WE 2), Obviously, the set S of all functions for which 
condition (2.5) is satisfied is an $-system, By what has 
been proved, $f contains the characteristic functions of all 
sets (2.6). The latter form a x-system generating the o- 
algebra Nt, By lemma 1.2, <? contains all \t -measurable 
P;,2 -Summable functions, Relationship (2,5) is now fully 
proved, If B is any set of JN, on putting f= 7, in (2.5) 
we get (2.4). 


Corollary. If A€of#i, BE Nt , then 


Ps, »(AB)= f Pt, 2,(B) Ps, « (de). (2.9) 
A 


If is Wi-measurable 7 o‘~measurable and fy Ps, ~Summable 
ther: 


Mg, 257 = Ms, x [&Mz, 2,1] *). (2,10) 
Expression (2.9) follows from a comparison of (2.4) and 
(1.11), and (2.10) from a comparison of (2.5) and 1.6.1. 
2.3. We put 
P(s. x; A; t,T)=P, 2 {[A, €P} 
(O<s<t, AC Ki, TE ). 
On the basis of lemma 1,6, we can write condition 2,1.F’ in 
the following forn, 
2.1.F7°, 0<s<t<uTE#. AC Mt, then 
P(s x; Ara Ti = [Pls, x; At dy) PQ ys wD). (2,11) 


K 


*)The functions #1 and &M., 27 are defined only on the set 2, 


In accordance with sec, 1.6, their mathematical expectations 


are understood to imply integration over the set ,. 
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We notice that, withi=2, P(s, x Aisa. T)=P(s, x; 41) and 
equation (2,11) becomes 

P(s, xia, T)= [P(s, x; tdy) P(t 95 4 YT) 

i | (2.12) 

O<s<ct<cu, TE F). 


This relationship for the transition function of the pro- 
cess is usually known as the Kolmogorov-Chepmen equation *). 


We put 
P(s, ity Te ee er be Pa) =P oo, o (Ht, ET «+++ X14, € Ph) 
2.13 
(KEE Ost, ...4 Tye. The #). ( ) 
Let s Db eiee Shy On putting 
A= (Xe, € qT. Sate Xt ET aa} t= tap 4th r=TD,, 
in relationship (2.11), we get 
P(x tT... ty PS 
= f PH ty Py ees baa GY) (tng Ji tye DD. (2.14) 


tT, 1 


Hence we find by induction that 


P(s, x, ty Ty. svt Py) = f- = fro, xi ty dy) X 
TT (2.15) 
X P(t M5 fer dp). 6. Pav In-v fr Ty) 
S<hidh<... C4 Ty... Tr€ #). 


Lemma 2,1, If of#f= AN? , condition 2,1.F in the definition 
of Markov process can be replaced by the requirement that 
(2.14) be satisfied for any n=1,2,...,.0<8 <4 cic... 
<ty qT, eee Ti€ #. 


Proof, It follows at once from (2.14) that condition 
2.1.F°’ is satisfied for any 
A= {Q,, x,ETy,..., 4, ET} 6 
(n= 1,2,...3t, ...,0,E[s, th (2.16) 


T,, eee T,€ #). 
*)On putting in particular ;=£, s=t=u and taking 2.1.5 
into account, p(s, +:5.£) is seen to equal 0 or 1 for any 
$30, x¢C6 
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Sets (2,16) form a x-system C in Q,, Om the other hand, 
the class ¢¥ of all sets A for which condition 2,1,F°’ is 
satisfied is a \-system (in Q,). By lemma l,l, since ¥ > @ 
we have $£26(@)=wW; , and the lemma is proved, 


Lemma 2,2, For any s>0, AGW P, (A) is a #-measurable 
function of x, 


Proof. We show by induction over n that all the functions 


P(s, x; tf) Ty ....t.T,) are Measurable with respect to x, Our 
assertion follows for »=1 from condition 2.1,.D. The passage 
from n—I1 ton is accomplished by means of lemma 1,7 and 


equation (2.14). 


We now remark that the class ¥ of all AE€W* for which 
the assertion of the lemma holds is a j\-system, By what has 
been proved, ¥ contains the x-system @, consisting of the 
sets 


A= {x1 ET ..-. 4%, EP} 
(n= 1, 2,...5 ty... 2, Ss 
T, ...,T,€ #). 
In view of lemma l,l, F20(@)=dW*. The lemma is proved, 


2.4. Let » be an arbitrary measure on the sc-algebra #, In 
accordance with lemma 2,2, the integral 


Ps, (A) = f Ps, o(A)p (dx). 
E 


has a meaning for any A€ dN. 
It may readily be seen that this integral defines a measure 
on the c-algebra off and that (2.3) [and therefore also (2.2)] 


remains true on replacing P, , by Ps.*). If »p is a proba- 
bility measure, P,,, is also a probability measure, and the 


*)It may easily be verified, on the basis of lemma 1.2, that 


Jf 700) Ps, 2(doyu (ax) = f f(0)Py, « (do). 
E@ rf 


for any of" -measurable bounded function f(w) . Expression 
(2.3) soon follows from this, 
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value P,; (A) is naturally interpreted as the probability of 
event A if the moving particle has a probability distribu- 
tion p» at the instant s, 


We write A€S* if, for every measure on the s-algebra 
8, an A, A, of sf? can be constructed such that A,S AS A, 
and P,,.(Ai)=Ps,u(Ao) « Obviously, di*co#* . The measure 
P,,, can be extended to the c-algebra ,f** in precisely the 


way the measuresP,,2 were extended in sec, 2.2 to the o- 
algebra cf’, 


Theorem 2,1", Expressions (2.4) and (2.5) remain valid fo 
any BEN and any f’-measurable P,,2-summable function §&, 
Expressions (2,9) and (2,10) remain valid for any A€oMi, , 
BES and any functions §, 1, such that fis of} -meésurab] 
n oft-measurable, and y-and 7 Ps,» -summable*), 

Proof, Let A€oM#:, BEAN. The expression 

pM)=Ps,2(A, ET) TE #) 


defines a measure in #, We choose 8, and B, from WW such 


that8,S 8 B,and Pp, ,(B) =P, ,(B,)- By the corollary of 
theorem 2.1, 


Ps, @ (AB;) = ff Py zy, (Bi) Ps, « (dw) = 
A 
= fe (ay) Pe, y B= Pr, »(B)- 
E 


It follows from AB, © AB AB, that 
Pt, 2 (Bi) = Py, 2 (AB,) < Py, 2 (AB) < Ps, 2 (AB,) = Py» (By). 


The extreme terms of this inequality are equal, so that 

Ps, @(AB) = Py, y (B;) = i Pt, 2, (Bi) Ps, 2 (dw). (2.167) 

Furthermore, it follows from B,<¢ Bc 8B, that, for all w€Q,: 
Ps, 2, (Bi) < Pz, a, (B) < Pe, x, (Be). 

The integrals of the extreme terms over set A with respect to 


*)We assume that all the measures P,, are extended to the 
onalgebra of* as described in sec, 2.2, 
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the measure Ps,» are equal, therefore 
Pix, (Bi) = Pu, «,(B) = Py, 2, (Br) (a,c, A, Ps, x). 


Comparison of this expression with equation (2.16’) shows 
that (2.9) is satisfied, Hence it follows, by the definition 
of conditional probability, that equation (2.4) holds for 
every BEAN. We deduce (2.5) from (2.4) by making standard 
use of lemma 1.2, Finally,(2.10), follows from a comparison 
of (2.5) and 1.6.3. 


Remark, As mentioned in sec, 2,2, the Markov nature of a 
process is not affected by replacing oft by of?. We can 
therefore write .4% for o#% in the statements of theorems 
2,1 and 2,1", 


2, Stationary Markov Processes 


2.5. Let oW* denote the minimal system of subsets of the space 
Q,={C> 0} that contains all the sets {x,€T} ¢>0, TE #) 
and is closed with respect to the operations of addition and 
intersection of any number of sets and with respect to the 
operation of taking complements, The Markov process X¥=(x, C 
oMi, Ps) is said to be stationary if a subset §,A¢Q can 
be associated with any />0 and any subset A€,\* such that 
the following conditions are fulfilled: 


2.5.4. 6,25 = 25 8,(A N B) = 8,AN 8,8; 


6, (U4) =Us.4.; (14) =10,4. 


(the index a rus through an arbitrary set of values.) 
2.5.B, Oe {x,ET}=(xn€T} (>, PEF). 
2.5.0, For any ACA = W(2,] Py (0,4) = Py, 9(A)- 


It may be observed that. if operators §, and %, satisfy 
conditions 2,5.A-2.5.C, the system of sets -£ for which 
§,A—6,A contains the sets (x,€I} (a>0, T'¢€ g) and is 
invariant with respect to all set operations, It follows 
from this that operators 6, are defined uniquely for the 
process X, The following properties of the 6, are readily 
derived from 2.5.A-2.5.C: 
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2.5.0. ON" = Nene 


2.5.E, If BEA, then 0,8¢ gf and Py, .(0,8)—= Py .(B)(we 
assume that the measures P,, are extended to oh, 88 indica- 
ted in sec, 2,2, and put Bed if Be W« and for every 
measure‘ in # there are sets 8, and B, of WN such that 
B,SBSB, and Po ,(B:)= Pp, » (Ba). « 


2.6.Let the function §(w) (~E2y) be j*-measurable, The 
sets (E(w)-=a} are mutually disjoint and form the sum Q,, The 
sets 6, {t (w) =a} are therefore also mutually disjoint and form 
the sum 9,2) = 2 . We put 6£(m)=a if w€0,{E—a} . Hence 
for every \’*-measurable function [(m) (w€2&,) there is an 
associated function 6,:(w) defined on the set 2, 


We can readily deduce the following properties of operators 
Oe 


2.6.4. For any numerical set I: 
6, {FET} = (827). 


(it is clear from this that the function 0,(@) is 6,* - 
measurable.) 


2.6.B, The necessary and sufficient condition for 7=46 
is that, for any a, 


0{E =} = {1 =a}. 
or that for any a, 
9, (5 > a} = {n> a}. 


ne #f=C—t (E&,), (This follows from 2,5.B and 
2.6.3). 


2.6.D. %xs=xn (BE d*). 


2,6.B, If f(t... %m---) is any function in the space R™ 
and 4, ...,5,,... are arbitrary d\*-measurable w-functions, 
we have 


Bef Ge esos Ene os DPOB ooo Oye «+d: 


In particular, the operators 6, preserve all the algebraic 
operations and the operation of passage to the limit. 
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2.6.F, For every o’-measurable function =: 
Mz, 2943 = Mo, 25- (2.17) 


The definition of operator 4,5, as also properties 2.6.4, 
2.6.D, 2.6,E and the first half of 2.6.B are readily extended 
to functions whose values, instead of being numerical, are 
points of any measurable space (£, #). (The main thing is 
simply that # should contain all sets consisting of a single 
point.) The functions &(w) can in this case be defined in 
some subset S¢€,\** instead of throughout the set 2, ( 0; 
being now defined in 98). If follows from 2,5.B and 2.6.4 
that 


2.6.6, 4 %,= Xan 


Theorem 2,2. Let (x, %, of, P,.2) be a stationary Markov pro- 
cess, Let Ocscr. If Bead, then 


Po,2 (8B) oft) = Po,2,(B) (2-0-2, Py.) (2.18) 
If § is S-measurable and P,,,-summable, then 
Mo, (84M?) = Mo, 2,6  @.¢. 2p Po,z)- (2.19) 


This is proved simply by comparing theorem 2,1° with 
expressions 2.5.E and 2,6.F. 


Corollary. If A€o#?. BEd, then 


Po, 2(A0,B)= f Po, x, (B) Ps, « (dw). (2,20) 
A 


If § is of}-measurable, +of-measurable and 7 and §6,7 Po2 - 
sumable, then 


Mo, a (€0;1) = Mo, « (Mo, 2,11)- (2.20°) 


2.7, We have already seen in sec.2,5 that the operators 6, 
are uniquely defined by the process X¥, The conditions will 
now be discussed in which we can assert the existence of 
such operators 6,, i.e, stationarity of the process x—(x,,¢, 


Aj, P. 2) « 


Theorem 2,3. The necessary and sufficient condition for 
the Markov process X= (xp, off, Py, 2) to be stationary is that 
it satisfy the following conditions: 
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2.7.4. Plh xs t+h, P)= PO, x; t,7. 

2.7.B. For any 0<i<t(w) there exists o’€2) such that 
2.7.B,. Sw) =C(w)—#. 

2.7.Bys tyn(o)=x,(0’) for all 0<A<C(w/)=S(o)—+t. 
Suppose that the process X is stationary. 


We shall write ©’ =c,;(w) if w’ and » are connected by 
relationships 2.7.B,-2.7.B,. Now 


6,4 = {w: cE A} (AE o*); (2,21) 


88 (w) == & (co) (2,22) 
tis an d*-measurable function, w€Q).) 
» w€2, 


Proof, Necessity. Condition 2.7.4 follows from 2.5.B and 
2.5.C. We show that condition 2.7.B holds, For every w€2, 
let A, be the intersection of all sets A€N\" such that 
w€%A , Obviously, A,E€N* and w€6,4, . We notice that, 
if w’€a, , conditions 2.7 .B,-2.7 Bo are satisfied, In 


fact, let “(w)=4, %4;,(¥)=a. Then 


WE (S=4, x1,, =a} = 0, {2 =u —zs, X,= a}. 
Therefore w/€A,S (2 =u—t, x,—=a} and {(w)=u—z, iar 


Sufficiency, We show first of all that, if A€d' , either 
all or else none of the values of “% belong to A. The class 
of all sets A having the property in question is in fact 
closed with respect to all set operations and contains (by 
2.7.B,-2.7.B,) the sets {x,€T} (4>0,TE€#). The aggregate 
thus Contains o*, 


It remains to show that condition 2.5.0 holds. Let ¥ be 
the system of all sets 4 for which 2.5.C is satisfied. Let 
C denote the system of all sets 

A= [x,€Ty es tn, €Ta}] O<m<... Shy). (2.23) 


Obviously, ¥ is a }-system and @ a x-system in the space 
Q. 


Moreover, for sets (2.23), we have 
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Po, 2(A) = P(0, 0; hy Py ees Ay Tye 
Py, 2 (0,4) = P(t. x; fy Ty... fey Py) 


We observe on comparing 2.7.4 and (2.15) that Po2(4)—= 
Py, 2 (8,4). Therefore ¥ 2@ and by lemma 1.1, F 33(@)= 


NO [Qo] 
Operators 6, thus satisfy all the requirements 2,5.A-2.5.C. 
Relationship (2.22) follows from (2,21) and 2.6.D. 
2.8.Let (x, 6, of’, P,, 2) be a stationary Markov process and let 
6, be operators satisfying conditions 2.5,A-2.5.C. We con- 
sider the system (x,,[, of, P,,.4,),, where sof,— of}; Pyp=Po, x « 
Evidently this system has the following properties: 
2.8.4. If t<cu and A€o#,, then [4,5 >a} € of, 
2.8.B,  {x,€T}€o%, ¢(>0,TE #) 
2.8.C, P, is a probability measure on the s-algebra of. 
2.8.D. Forany t>0,FE8: 
P(t, x, T)=Pz{x,€T} 
is a #-measurable function of x, 
2.8.B. P(0,*%,E\x)=0. 
2.8.F. For any ‘>0, ACW 
P, {8,A] AM;} = Px, (A) (a.c ° 2s, P.,)- 
2.8.6.  §,2,—=2,, 0,(A\B)=6,A\6,B, 
6, { U A,} = U 0,4, 8, { n A,} = n 944, 
(a runs through an arbitrary set of values). 


2.8.H. {x,€T}=(%,,€T}  (¢>0, 4>0,TE g). 


Theorem 2,4, Given: 
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a) a function C(w) (w€Q) taking values from the interval 
[0, + co]$ 


b) a function ~;,(w) defined for w€Q, ¢€[0,C(w)) and taking 
values from the measurable space (F, #) 3 


c) for any t>0 a c-algebra of, in the space Q,— {C>1#}; 


a) for each x€E a function P.(A) in a c-algebra of in 
the space © containing of, for all ¢>03 


e) for each ¢>0 and A€WN* aset %ASQ (the system 
dW is defined in accordance with x,(w) and © as in sec. 2.5). 


We suppose that the system (x, °, of, P,, 6,) satisfies condi- 
tions 2,.8.A-2.8.H, Then there exists a stationary Markov 
process X=(%1,5, oft, Ps.) for which of;=of, Po=Po,2 
and 6; is a system of operators satisfying requirements 
2.5.A-2.5.C. 


Proof, We put ofj=of, Po2z=Pr. With s>0, let of® 
denote the class of all sets of the form §,A and 8,u0,4> 
where ACW, and let 


Pec (6,4) =P, (A), 
Py, 2 (2 U 9,4) = 1 —P, (2p) + P(A)» 


of; = 0,N,_,"). 
It may easily be verified that the system (x, 6 ofM%, P, .) 
possesses all the properties 2.1.A-2.1.F, whilst the opera- 
tors 6, possess all the properties 2.5.A-2.5.C. The only 


property whose proof requires a certain amount of working is 
2.1.F. 


To start with, a function 0, can easily be defined for 
every '-measurable function as in sec, 2.6. Properties 
2.6.A-2.6.G are soon seen to hold here. 

We have to verify that, for any s<t<u, TE #. ACK, 

P, (A, x,€T} = [Pa X23 4, T) Py, 2 (dw) = (2,24) 
Fi . 
= Mg olt,P (h xy5 4 TY. 
*) i, denotes the :-algebra in %={£>/} generated by the 


sets (.€l, > O<ud<t Teg), and ‘NY the s-algebra in 2 
generated by the sets {*,¢!} (430, Pe #). 
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This relationship is obvious if s=—0. If s>0, we have 
by definition of :A=0,8, where BCod,,. By 2.8.G, 2.8.H, 
2.6.D and 2.6.5, 


o, {B, Xy-a ET} = {A, x, ET}, 
9, [xeP (t+ 5 2 T)] = 4,P (tx, aT) 


and by 2,.5.C and (2.17), relationship (2.24) is equivalent to 
the equation 


Pz {Bs %y_g€ TP] = Ma xpP (t,x, 93 as P= 
= f Pt. x54 T)P, (dv). (2.25) 
B 


To prove (2.25) » We only need to verify that 
Pz (Xue € Do%,_.} =P, 453 4 T) 


. 


But, by 2.8.F and 2,.8.H, 


Pz {%y-eE To4,_s) =P, {8,_[%y_1ETIfo4;_,} = 
= Po,_, {*y-2ET} =P(u—t, X59 T)(a.c 2h» P,). 


It remains to observe that, by 2.5.C and 2.8.H, for every 
JyEE 
PO ys ws T)= Pay (xu€T) = Py (x, 1€T} =Pu—t, y. 1). 


The theorem is thus proved, 


We shall employ the term 'stationary Markov process" below 
in two different senses, either for the system (x, ¢, offi, P,,.) 
subject to conditions 2.1,A-2,.1.F, for which a system of 
operators 6, exists satisfying requirements 2.5.A-2,5.C, or 
for the system ‘x,, [, of; P,, 0,) subject to conditions 2,8 .A- 
2.8.H. Writing x=(x,(.o#j,P,,,) denotes a stationary Markov 
process in the first sense, whilst X’ = (x, 5 o&, P,, 9,) denotes 
a stationary Markov process in the second sense, If of =of,, 
Po,2=P, and operators 9, satisfy conditions 2.5,4-2.5.C in 
regard to the process xX, we shall say that X corresponds to 
X’ (or X’ corresponds to X) and write X<~>x’. For every 
stationary Markov process in the first sense there is a 
corresponding stationary Markov process in the second sense, 
The converse is true by theorem 2.4: for every stationary 
Markov process in the second sense there is a corresponding 
stationary Markov process in the first sense, The two 
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meanings of the term "stationary Markov process" are thus 
very closely related, The difference between them amounts 
to this: when it is a question of a stationary Markov pro- 
cess in the first sense, our knowledge of “*t ©. off, Py, 2 

is essentially complete, whereas when it is a question of a 
stationary Markov process in the second sense our knowledge 
of <x,,5, Ai, Po, x % is essentially complete but there is some 
arbitrariness in the choice of s-algebras of; and measures 
P,,2for *«>0. 


3, Equivalent Markov Processes 


2.9.Let X=(xp 6, of}, Pe, x) and X=(x, © offi, Pe, x) be two Markov 
processes in the same phase space (Ff, #). We shall say that 
X is got from X by means of the transformation of the space 
of elementary events 7: +2 (og! is the space of elementary 
events for x, © the similar space for ¥, and Y the mapping 
of & into 2), if the following requirements are satisfied: 


2.9.4. T@)=t[y(@)) Wwe). 

2.9.3. H()=417@] GER, 0<t <Fo)=C1y @p. 
2.9.0. Mi=1-' (chi) *)- 

2.9.D. of’=y-"(of') and P,(1-14)=P,,(A) (AE of). 


Evidently the process X is uniquely defined, given the 
process X, by the transformation 7. Now what conditions 
must be imposed on the mapping 7 for a Markov process satis- 
fying conditions 2.9,.A-2.9.D to exist? The answer to this 
question is supplied by the following theorem, 


Theorem 2,5. The necessary and sufficient condition for a 
mapping +:@2@ to give a transformation of the space of 
elementary events of a process X is that: 


2.9.0. For any s>0, x€E, A€o#’, it follows from 42 (8) 
that P,,.(A)=1**), 


*);-1(A) denotes the complete pre-image of A for the mapping 
Y, i.e. the set {oty(aj)ce4}. If e is a system of subsets of 
the space 2,7-1(.4) dénotes the system of all sets 1-1(A) (A€ 4). 
**)In other words, no matter what measures P,, We start 
from, the corresponding outer measure of the set 1(@) is 
equal to unity. 


MARKOV PROCESSES 43 


Proof, If A218) » then 7-'(A)=9 and by expression 
2.9.D | P,.7(A) =P, ,(@)=1.. We have thus proved the necessity 
of the condition. Turning to the proof of sufficiency, we 
consider the elements 7, %,, of#j,o#° and P,, 2 defined by expres- 
‘sions 2.9.4-2.9.D. We show first of all that the function 
P,, 7(B)is defined uniquely for any B€o#*, This is done 
simply by verifying that, if 


B= 1-1(A) = 771(A)s (2.26) 


then 
Ps,» (A) = Ps x (Ap). 


In fact, it follows from (2.26) that 
Th AN Ard) = TEADN AD TA) = 8 


Consequently A,\A,4,27(8) and by 2.9.2, Pse(AN\dA)=1 
and therefore Ps,o(Ai\4,A4.)=0, Ps. (Ar) == Ps,_(Atdo)« It may 
similarly be shown that Py (Ap) = Ps, (AAs) « 


The proof of the fact that the system (x, %, off, B, ,) 
satisfies conditions 2,1.A4-2.1.E and 2.1.F” presents no 
difficulties and is left to the reader. 


We shall teke two particular cases of transormation of the 
space of elementary events. 


a) If 7 is a one-to-one mapping of & into 2, & can be 
identified with some subset of set ©, The passage from 
process ‘X to process X reduces in this case to restricting 
the domains of definition of functions {(w) and x,(w) , to 
replacing each set A of of*(or of of?) by its intersection with 
Gand to a natural transfer of measures P, . to these inter- 
sections, We shall speak in this special case of the pro- 
cess X being got from X by a restriction of the space of 
elementary events. Given xX, the process X¥ is uniquely 
defined by the set §. By theorem 2. 5, this latter is sub- 
ject to the single requirement: 


2.9.0’, For any s>0, x€E, A€o#, it follows from 42 
that P, 2 (A)=1 “ 


b) If 1 is the mapping of @ into @, each point »€Q can 
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be visualized as being "split" into the set of points | 
y-1(w)={0:7(0)=0}, In this case we shall say that X is 
got from X by splitting the elementary events. 


It may readily be seen that a general transformation of the 
space of elementary events can be got by successively purging 
the space of elementary events (when @ is restricted to7 (8) 
and ee the elementary events (signifying passage from 
1@) to @). 


2.10. Let X=(xp, 6 oMf, Pen) and X= (x, 6. oft. P, .) be two 
Markov processes in the same phase space and with the same 
space of elementary events, and let 


2.10.4. off 2 of’. 
2.10.B. of of? and P,,,(A)=Ps,2(A)(AE oft’). 


~ 


We shall speak here of X being got from X by a widening 


(or of X being got from X by a narrowing) of the basic o- 
algebras, 


The operation of supplementing the measures described in 
sec, 2,2 can be quoted as an example of widening the basic 
¢~algebras, As an example of narrowing these algebras, we 
may mention the passage from‘s-algebras of%, o#* to c-algebras 
AN%, A, which was also discussed in sec, 2.2, 


2.11.We shall speak of the Markov process X being subordin- 
ate to the Markov process X if X¥ can be got from X by 
carrying out successively a transformation of the space of 
elementary events and a widening of the basic s~-algebras. 
The necessary and sufficient condition for this is that some 
mapping 7:9 Q satisfy the following requirements 


2.11.4, U)=C17@)} @E®) 

2.11.3. %,(0)= x[1 (0)] (ER, 0<t < F(a). 

2.11.0. off Dy-* (off). 

2.11.D. of’ Dy-1(0M*) and B, (7-14) =P, . (A)(AE of). 

If process X is subordinate to process X, every traject- 


ory of X is at the same time a trajectory of xX, The trans- 
ition functions of processes X and y¥ are the same, 


MARKOV PROCESSES 45 


We shall consider every possible interval [0,4) and every 
possible function ?() defined in these intervals with values 
in the space E, The class of all these functions will be 
denoted by 2,. Let ¢€2, . We shall write %(») for the 
right~hand end of the interval in which the function 9 is 
given, Two elements ° and } of 2, will be reckoned identical 
when and only when {(¢)— f() and o()=¢() for all 


0<t<li@=lw *)- 

We put ~(9)=9) O<t<%(~)) and write &” for the o- 
algebra in space ‘2; generated by the sets (9: x,(9)€ r} 
(w>s, TE gw) and AN; for the c-algebra in space {y; Ce) >t» 
generated by the sets [7:xi(9)ET, C(2)>2} (Els, th TE@) » 
The Markov process (x, 5, o#j, P, .) in the phase space (E, ) 
will be said to be canonical if we have for the process 


2=ek,, =<): t=, oi = N° ’ off = N". 
Lemma 2,3. Every Markov process X=(x,,C, of8,P, ,) is sub- 
ordinate to some canonical process yx — (ep & ws, P ‘ 
> ee I 8,2. 


Proof. Let 2 be the space of elementary events of the 
process X, We associate each »€2 with a function (4 
having values from E£ and defined in the interval [0. €(w)) 
by the expression 


PO=%(0) (Ot <C)). (2.27) 


Expression (2.27) defines the mapping 7:22, **), 
We evidently have here 


C1 (@)] = (@), 
X4 [7 (w) ] = %,(w), 
11M) & of, 7-1 (ol?) SoM. 


For every A€d\* we put 
Py, (A) = Py, o(q~1A). 
The system (x, ¢, of, P, ,) satisfies conditions 2.1,A-2.1.F 


*)One of the elements of the space 2, is the function ¢ 


which is nowhere defined, For this, £(%,.) > 
**)If %(0)=0, we put y= , where % is the element of 
22 defined in the previous foot-note. 
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and therefore yields some Markov process x . This process is 
canonical and the process X is subordinate to it. 


The canonical process X constructed in lemma 2.3 will be 
called the canonical form of the process X, 


Remark. All the concepts and results given in sec, 2,9- 
2,11 for Markov processes may be carried over directly to 
Markov ‘random functions, In particular, we can assert the 
following, which will be useful later. 


The necessary and sufficient condition for the space of 
elementary events of the Markov random function X= (x,, %, of, 
P) to be reducible by restriction to the subset G=q is 
that, for every aA¢€o#, it follows from 4G that 
P(A4)=1. 


2.12.We shall speak of the Markov processes X’ and X” being 
equivalent if they are defined in the same phase space and 
have the same transition function, 


Theorem 2.6, Every class of equivalent Markov processes 
contains one and only one canonical process X and consists 


of all the processes subordinate to xX, 


Proof, By lemma 2,3, it is a matter of proving that, if 
two canonical processes X’=(x,, 0, AN, Pj.) and X”=(x, 6 
WN"? Py, 2) have the same transition functions they must coincide, 
We thus need to show that, for any A€ A", 


Pin (A) =P¥, (A). (2,28) 


By (2.15), this relationship is satisfied at least for the 
sets 


A={x,€T, Brea %1,€Tn} 
(ml, Ws Shy cc tes Dy eee TER) (2.29) 


The sets for which equation (2.28) is satisfied form a i- 
system F, whilst the sets of form (2.29) form a t-system @, 
By lemma 1,1, it follows from ¥ BC that ¥F Do(@)=—dN". 


We shall discuss in detail in Chapter 4 the question of 
what sort of functions are transition functions for Markov 
processes, 


MARKOV PROCESSES AT 


2.13.Let L be a subset of the set Q,. Let A py denote the 
aggregate of all Markov processes corresponding to the transi- 
tion function P(s, x; t, T) and such that all their trajectories 
belong to the set &, The aggregate o% py may be empty, If 
it is non-empty, we can conclude on the basis of theorem 2,5 
that: 


a) the set 2 satisfies condition 2.9.« with respect to 
the corresponding transition function P(s, x; ¢, [) for the 
canonical process X 3 


b) the space of elementary events of the process X can be 
contracted by means of a restriction to the set 2, 


. We shall describe the Markov process obtained as a result 
of such a purge as -canonical. The following variant of 
theorem 2.6 admits of a simple proof which we shall leave to 
the reader. 


Theorem 2,7. If the class KH p¢gis non-empty, it contains 
one and only one -#-canonical Markov process | X and consists 
of all the Markov processes subordinate to X. 


In particular, on choosing as 2 the class E" of all 
functions defined in the interval /=[0, 00), we arrive at 
the following result. 


Corollary. Every class of equivalent non-cut-off Markov 
processes contains one and only one canonical non-cut-off 
process X and consists of all non-cut-off Markov processes 
subordinate to X, 


We shall now prove an important theorem which enables us 
to judge whether some class of equivalent non-cut-off Markov 
processes contains a process whose trajectories possess cer- 
tain previously assigned properties. 


Theorem 2,8, Let X=(x» oMf, Ps,z) be a non-cut-off*) 
Markov process in the phase space (F£, g) and let .%¢ £7 
We shall assume the existence of a non-negative function 


Gti XH ever by Lys vee) 
(OS by cer bye 005 Myr oer qr ee EE), 


*)A similar theorem for cut-off processes is proved in 


sec. 6.1. 
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which, for any s>0 and for any denumerable everywhere dense 
subset {t,,....f,..--) of the interval [s. co) isa #”-~- 
measurable function of «:,.-- *,..., and satisfies the follow- 
ing conditions: 


2.13.4. If Gf 1, .... ta x, ...) = 0, there exists Cp 
such that 9 (t,)=x, (R= 1,2, ...,4,...). 

2.13.B. Mg, 2G (bie Xt ++ Ene Hye --) =O (or what amounts 
to the same thing, Py 2 {7 (fi Xp. -- +1 fae Xp ++) FO} = 0). 


Then there exists for the process X an equivalent Markov 
process, all the trajectories of which belong to the set 7, 


Proof. We shall take the canonical form of the non-cut- 
off process X: X=(x;,, NY, P,,). The theorem will be proved 
if we can show that condition 2.9.a is satisfied for process 
® and set Y, i.e, that it follows from ARDY that P,, 2(A)=1 
no matter what s>0, xCE and AE W*. 


Thus let A€cV* and AD.Y. On the basis of lemma 1.5, 
we can write the characteristic function of A in the form 


La OD=S ty (Qe oe Hig (Pe DS 


2.30 
= Stel «0 el 1 EES, (+30) 
where 1#;,..-.,f,,... is some sequence of points of [%, 00) . 
We can obviously suppose without loss of generality that 


ty, ....t,,... is everywhere dense in [s, co), 
We show that, if p€A , then 
Gt» P(tr), .. 0s tar Vea)». ] HO. 
If in fact we had 
Gite VED. «20s ty OE)» «J =O, 
there would exist a function ?€.% by 2.13.4 such that 
o(4)=e() fork=1,....2,...By (2.30), “a =X4(9) , and 


%€A inasmich as 7€.YSA ., It now remains for us to 
observe that, in view of the equivalence of X and x, 


Mao Op Biss sachin eyes 0 MG Ole Misscesty esc) M2631) 
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(this is proved by conventional use of lemma 1.2). 


It follows from (2.31), 2.13.B and the fact that the func- 
tion 7 is non-negative, that 


By 2 (ahs, Xt,» sesstyy Xe, +.) 0) =1 
so that all the more, 
P, 2 (A)= 1. 


Remark, Let all the trajectories of the non-cut-off pro- 
cess X belong to the set SEF! and let the function 
G(ty, Xp ++ ty Hy, ...) Satisfy 2.13.B and the following 
condition: 


2.13.4". If b€.% and (fy (4), --.5 tae OQ). =O , the 
function 9€.N.% exists such that 9 (f)=4 (4). 


Then there exists an equivalent of xX, all the trajectories 
of which belong to 9N.%.. 


In fact, let X be the canonical form of process X. Let 
A€d and ADEN, . We put $68 if 7h (h). ....t, 
b(t,),...)40.+ We show, as in the proof of theorem 2.8, that 
PNASB Therefore .#)SAUB8 and by theorem 2,5, 

P, 2(AUB)=1° By condition 2,13.B, P,,.(B)=0 , so that 
P,, 2(A)= 1. 


2.14. We shall give an illustration of the application of 
theorem 2.8, 


Let T and G be measurable sets in the space (£,9), We 
shall speak of T being inaccessible from Gif, for any 


0csct 
{%(w) EG} & (x,(w) ET}. 


If set T is inaccessible from ENT, we shall simply say 
that Y is inaccessible, 


Lemma 2,4, If the transition function of a non-cut-off 
process X satisfies the condition 


2.14.4. Pls, x 4 T=0 for all x€G, 0<sct , & process 
equivalent to X exists for which the set I is inaccessible 
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from G, 


Proof, We write % for the set of all functions ¢(2) 
(0<t< oo) for which the following condition is satisfied: if 
g(s)EG for a certain s, then (ET for any t>s., By 
theorem 2,8, our assertion will be proved if we can construct 
a function’ (ty xy...) ta Xn. Yt; 0, «;€ E)that satisfies condi- 
tions 2,13 .A-2,13.B. 


We put 
Gtr, Mas een te Mae) == SM Xe (%) Zr (*)). 
B<tj 


Obviously, the function 7 is @«-measurable and non-negative 
and satisfies condition 2.13.A. Furthermore, 
Mg, 29 (fi tyr sees bn tyr + j= 
= »> Ps, 2 {i € G, *1,€T) = 
ti<ty 
=) [PG x t dy) PG, yt T= 0. 
t.<t,¢ 
Thus the function g also satisfies condition 2.13.B. 


2.15. The following theorem is also useful in investigating 
the properties of the trajectories of Markov processes, 


Theorem 2,9, let X=(x, 6, off, P,,) be a Markov process 
in the phase space (F, #) and let £ be an arbitrary subset 
of space 2... We suppose that there is associated with each 
pair s>0, x€E a set 2°*-@, having the following properties: 


2.15.4. For each w€2** there exists a function ¢(s)(#€[0, 
)) of # such that .=—C(w) and ¢()=-x,(w) for all f€ls, 


5 (w)) + 
2.15.B. If A€o#* and ADQ*® , then Ps, o(A)=1. 


Then a Markov process X equivalent to X exists, all the 
trajectories of which belong to -7, 


Proof, We put 


C® 7 (w) == max (C(w), 5), 
x97 (w) = x,() (FE[s, 90) o€ Q)). 


In accordance with sec, 2,1, the elements (x?”, 2°, of, Ps, x) 
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define a Markov random function X°” in the time interval 

Is. co) . Let X%* denote the Markov random function obtained 
from X** by a restriction of the space of elementary events 
to the set 2°” (by the remark at the end of sec, 2,11, the 
existence of such a function is guaranteed by condition’ 
2.15.B). The system X*” (s>0,x€EF)is easily seen to be a 
Markov family. By using the construction described in sec, 
2.1, we can build up from this family a Markov process X in 
such a way, (in view of condition 2.15.4) that all the traject- 
ories of X belong to &, It may readily be seen that X and 
X are equivalent, 


2.16. We shall dwell in conclusion on some special problems 
connected with the subordination and equivalence of Markov 
processes, 


Let X=(x1 6 oMt, Ps,z) and X=(%p © of, Bx) be stationary 
Markov processes, the process X being subordinate to X, 
Let 4, and 4% be the corresponding operators satisfying con- 
ditions 2.5.d+2.5.C, Then : 


Byy-1A4 = 7-18,4 (AEN). (2,32) 


For, 
17? (%, ET} = (%, 7). (2.33) 


The events A , for which relationship (2,32) holds, form an 
invariant system with respect to all set operations, By 
(2,33) and 2.5.B, this system contains all the sets [x,€T}, 
It thus contains "yr ae 


Theorem 2,10, Let e% be the class of equivalent Markov 
processes having the transition function P(s,x;¢,T). The 
necessary and sufficient condition for class oto contain a 
stationary process is that 


P(s, if TV=PO.x; t—sT) O<s<t x€E, PER). (2.34) 


If condition (2.34) is fulfilled, the canonical*) process X 
belonging to of is stationary. 


*)As is clear from the proof, any . -canonical process is 


stationary, provided only that 9 satisfy condition 2.7.B. 
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Proof, The necessity of condition f2,.34) follows from 
2.5.B and 2.5.C, On the other hand, the canonical process xX 
always satisfies condition 2.7.5. If (2.34) is satisfied, ¥ 
also satisfies condition 2,7.4 and, by theorem 2,3, is a 
stationary process, 


2.17. We shall now look on a stationary Markov process as a 
system (x, (, of; P,. ,) satisfying conditions 2,8.A-2.8.H, We 
shall describe the stationary process X=(x, & C, Ay, P,., &) as 
subordinate to the stationary process X= (x,,¢, of, P,, 6,) if 
we can construct a mapping y of the space of elementary 
events & of process X into the space of elementary events 
@ of process X such that: 


2.17.4. T@=ir@)) (we). 

2.17.3. %,@)=x,17 (01 (EB, 0<t <F@). 

2.17.0. 1-0, S of, 

2.17.D of Dy-'o0 and B,(7-'4)=P,(A) for AG o#*). 

Theorem 2,11, The process X= (x, © of, P P,, 6 8) is subordin- 
ate to process X=(x;,,¢, of; P,, 4,) When and _only when there 
eee stationary Markov processes X = (x, F t off, P P,, a) and 


X’ = (x, 6, off, Py.) Such that X’~—>X, x<»x and x is sub- 
ordinate to xX’, 





Proof. The sufficiency of the condition is obvious, The 
necessity is proved with the aid of the construction used in 
proving theorem 2,4 (sec. 2.8). 


We shall say that a process X’=(x,,%, of, P,, 0, is canonical 
if x’<>xX, where ¥= (x1, 5, offi, P,, 2) 18 a canonical process 
in the sense of sec, 2.11, Two processes X= (x, 5, of, Pz, 9,) 
and X= (x, f, F, of, P,, 6,) Will be called equivalent if their 
transition functions coincide, i.e. for all 120, «EF, TEM 


P, {x ET] =P, (x€T). 
Theorems 2,6 and 2,11 readily yield the following result. 
Theorem 2,12. -Every class of equivalent stationary Markov 
processes X = (x, 5%, of, P., §,) contains one and only one canonical 


process ¥—(x,,%, of, P,. 6) and consists of all stationary 
processes subordinate to xX, 


*)Relationship (2.32) follows obviously from these conditions. 


CHAPTER 3 


SUBPROCESSES 





1. Definition of Subprocess, The Connexion Between 
Subprocesses and Multiplicative Functionals 


3.1. We discussed in article 3 of Chapter 2 some construc- 
tions (transformations of the space of elementary events, 
widening of the basic a-algebras ) whereby further Markov 
processes could be built up from a given process, All these 
Markov processes have the same transition function as the 
initial process, 


The present chapter is concerned with a type of transforma- 
tion in which the transfer function is altered, The essence 
of such transformations consists in a shortening of the life 
© of the process, 


Let X= (xp % oft, Py, 2) and X =(,, %, offi, P,,z) be two Markov 
processes, where F(w) <6(w) for any € @, Xi (w) = x,(w) for 
0<t<F(w)and Aj = of? (8) (S.=(C>t}) . We shall speak in 
this case of X being got from X by a shortening of the 
life. 


We shall speak of a Markov process x being a subprocess 
of Markov process X if X can be obtained by shortening the 
life of some Markov process subordinate to xX. 


We note that every trajectory of subprocess X is part of 
some trajectory of process X, Hence, if process X is 
given in a topological measurable space and is continuous or 
continuous from the right*), the subprocess X possesses the 
same properties, 


Let 2 be the space of elementary events of process 


*)Process X is said to be conticucas if all its trajectories 
are continuous, i.e. if for every wé2 x,() is a continuous 
function of ¢ in the interval [0,¢@)) . Continuity of X 
from the right is similarly defined. 


oy 
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X=(xnk hi, P,, z) and ® the space of elementary events of 
process ¥—(x,, %, ofj, B,,,). In view_of sec, 2,11, the neces- 
sary and sufficient condition for XL to be a subprocess of 
X is that there exist a mapping 7: 8.2 satisfying the 
following requirements (cf, 2,11.A4-2.11.D). 


31a, 2 (eo) <Clr@)] (EQ). 

3.1.3. HO@)=x lr] CES, 0¢¢ <2 (0)). 

3.1.0. If A€oMi, then {1-1(A), C>t€ off}. 

3,1.D. of Dy-1 (A) and B, , (7-14) = Py, , (A) for AC of, 

We agree to identify every function {(w) in space 2 with a 
function §[y (s)) in space fs) » every subset A of space 2 
ss the subset 7-14 of space 9, and every system of subsets 

= {A} of space 'Q with the system of subsets 171 = {7-14} 

o space &. In these circumstances conditions 3,1,A-3.1 D 
can be rewritten as 

3.1.A°, <6. 

3.1.3. M= ty O<t<F. 

3.1.0°. If A€oM}, then {A,% >t} € of. 

3.1.D/. of Dof* and BP, ,(4)=Py,o(A) for AC of’. 

Our definition of subprocess is so far unnecessarily 
general, We narrow it down by requiring the following condi- 


tion to be satisfied in addition. 


3.1.5. For every x€E and any 0<s<t 
Pi o{S>tloh'}=ai (a,¢.2,, Py, 2) *) 
*)This relationship can be written in more detail as 
By a (P>ebrtem) sett @)] — (a.c.17 1, B,, ,)- 


In the formula in the text, y-“a* is equated with of, 1-1, 
with %, measures By a on y-oa8 with measures Ps,x on 
#° and finally, functions «{[y(@)}(0¢5) with functions «3 (w) 
(#€2). 
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where az(w) is an ys-measurable function*), 


The last condition may be pictured in the following way. 
When calculating the probability of the event 7>#,a 
knowledge of the total aggregate of phenomena connected with 
process X and observed during time [s, cc) yields no more 
than a knowledge of the trajectories of the process during 
time Is. ¢], In other words, given a knowledge of the course 
of X during time [s,f] an event @>t# is independent of the 
remaining phenomena observed during time [s,co) . 


We shall put A€&; if Ac", AGQ, and there exists BE? 
such that p, .(4\AB)=P,,2¢(B\AB)=0 for all x€E, 
Obviously, &; is a c-algebra in space ©,, It may easily be 
verified that condition 3,1,E is equivalent to the following: 

3.1.E’. For every x€E 

By,a (0 > tfoM|=ai(v)  (a.¢, Q, Pye)s 
where aj(w) is an &{-measurable function of w**), 


We note that it follows from MDc#AiawWi HAAN SD Si, 
A DRDAN; and condition 3.1.E that, for every x€E , 


By, c {2 > tlof'} =P, 2 (E> t]oAi] =P, (F> tp M4] = 


ae JB Ie ‘ (3.1 
=Py,0 {2 > t1Ri} =P. 2 {7 > efi} (a.c.2), Psa). (3-1) 

We notice further that, for any 0<¢s<4, x€E 9 
B,, a{S>t]oM} = (a.c, 2, Py, 2): (3.52) 


*)Condition 3.1.8 presupposes that function «jis independent 


of x. It would be sufficient to require that (+) be an 
Nix @ -measurable function of » and x, For in this case 
the function «}(, x,(#)) is of’j-measurable, does not depend on 
x and for every x differs from «j(w,x) only on a set of 

P,,2 measure zero. 

**)We put Aci? if, for every measure » on the s-algebra # 
we can choose 4;, 4» "of WN such that 4,644, and Ps,p (Ay) 
=Psu(Ao). We put acoso if syed? for every o>t, We 
leave it to the reader to show that the whole of ow theory 
of subprocesses remains valid if the o-algebra #: is defined 
by st=WVt+oNe?. The set of subprocesses is considerably 
widened with this definition and includes several important 
new examples, 
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For, since {{>}Ed\, we have from 3,1,A and 1.6.3: 
B,, 2 (Z> tof] =P, 2 {C >t T> tfos'} = 
=Zo,Ps,2(8> thot} (ac, 2. Py, 2). 

3.2. Lemma 3,1. Let X= (Xt %, offi, Pz) be a subprocess of 
Markov process X=(x;,,C, of, Ps,2) and let functions a} be 
defined by condition 3.1.E (or 5.1.B’). Then for any BE of" 

By, 2 (B. > t} = Ms, 27 pats (3.3) 
for any of-measurable function ‘ 
Ma, 28x 5 p= Me, otal. (3.4) 
The transition function of subprocess X is given by 
P(s, x; t, T) = Ms, a8tty (*2)- (3.5) 
Proof, By 1.6.F and (3.2), 
M, ele > t =*M,, x (eB, az (f> tfo#")} = M,, ott = M,, at 


and expression (3,4) is proved. On putting '=y, in this, 
we get (3.3). We remark further that, by 3.1.B, 


(xET} = (EP, E> 4}. (3.6) 
Expression (3,5) follows from (3.3) and (3.6), 


Lemma 3.2. The necessary and sufficient condition for two 
subprocesses 


X= (x a Ai, P,, a) & X= (x1, is Ai, P,, 2) 
to be equivalent is that, for any 0<¢s<t, x€E, 
B,2 (E> tfo#} =P, {E> tho#’} @.c.2,p,,). (3-7) 


Proof, The sufficiency of (3.7) follows directly from 
lemma 3.1, The proof of necessity reduces to showing that 


~. 


at == of (a.c, 2 Pr, 2) (3.8) 


where @! and @f are Wi-measurable functions corresponding to 


SUBPROCESSES 57 


subprocesses X and X by virtue of condition 3.1.B, We 


remark that, for any s<i,<t,<...<t,=47T,,..., CH vy 
(3.3) and (3.6): . 
Py, 2(s, x; ty, Yt, seer Lye T,) = P.0{x, ET, ores %,, € Pa} = 
= Poof, ET, -.., x4, Py i>4= 


= M,, 2 [Zr,(*,) --- tn, (*,,) a]: 
Similarly, 


P(s. #5 tT ++. ty T= M,, oftr,(4,) + Xr, (%:,) 2} 


But_it follows by (2,15) from the equivalence of ¥ and ¥ 
that Bis, x; ty. Ty... tye Ty) =P(S, x; ty Ty, +s ty T,)y and there- 
fore 


M,, al(@ — at) kr, (*,) +++ Xr, (*:,)]= 0 (3.9) 


We write for the class of all functions {(w)(w€Q,), @ 
for the class of w-sets of the form 


{ET aoe xt, ET,,} 
(a=1, 2,...58<¢4<.-.<t,=¢; 1, .... LE) 


and 3 for the class of functions é(w) (w€Q,) such that 
M,, » [(@ —@) ]=0. 


Obviously, C is a z-system, and d¥ an .2-system in space 
Q,. By (3.9), 58 contains the characteristic functions of 
all sets of @, and by lemma 1.2, S contains all functions 
measurable with respect to s(@)=W\{. In particular, # 
contains «{—at, Thus M, 2[4—ap=0 so that relationship 
(3.8) is satisfied, , 


3.8, The subprocess X= (Zp ©. off, Ps,2) of a Markov process 
X=(x, 6, cM, Py2) will be described as canonical if: 





3.3.4. The space of elementary events & of process X is 
connected with the space of elementary events 2 of process 
X as follows: = X/, where /=[0,-too] and 7(w, \)=w 
(WER, EN, 


3.3.B. 2(w, 4) = min [£(w), A]. 
3.3.0, %4(%)=x,(v) for 0<t<liw i). 
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3.3.D, Mi consists of all sets of the type AX(4.co], 
where A€ off. . 


3.3.E, of is a c-algebra in space g, generated by the sets 
(A, O >t} =14.2> 4] XG co] (AE of, f > 5). 


Theorem 3,1, Every class of equivalent subprocesses of pro- 
cess X contains precisely one canonical subprocess X and coin- 
cides with the class of all Markov processes subordinate toX, 


Proof, a) We show first of all that equivalent canonical 
subprocesses coincide, Let ¥= (xy, 7 ” ofi, P,, x) and X— (x, & zr 
oA’, Px, 2) be two equivalent canonical subprocesses of process 
X, By 3.3.A-3.3.E, these processes have the same space of 
elementary events Q X/ and we have for them T— 7 %,—-%, 

= Ai and AP = ht". 


It remains to show that, for any B€of*, 
P,, (8) = P,, 2 (B)- (3 -10) 


The sets 8 for which (3.10) holds form a )-system ¥ in the 
space S—=Q9—2x/. On the other hand, sets B=[A, ° AX 
(t, co] (AGoM’, t>s) form a t-system C, where «(@)=o% . By 
lemma 1,1, it is a question of proving "that FPC,i es 
that (3.10) holds for sets B=[A, S>¢] X(t, co] (AEM, ts). 


By (3.3), we have with any A€oM#*, t>s amd x€E, 


Bn (l4,5>4X@, col] =P, , 
B. 2 lA. S>t]X ( col} =P, 


.{ > th= My o/h 420 
ek > i= My hate 


where as and ag are functions satisfying condition 3.1.E for 
X and Fa respectively. Since subprocesses X and X are 


equivalent, (3.8) is satisfied and therefore 


Bc As S> 4X G co]] =P, (IA, 5 > XG Sol}. 


b) Obviously, if X isa subprocess of process X, all the 
Markov processes subordinate to X must also be subprocesses 
of X and equivalent to ¥. 


It remains to show that every subprocess X=(x,, ~ off, Ps, ow 
of process X is subordinate to some canonical subprocess x, 


Let us suppose that the space of elementary events © of 


subprocess Y is connected with the space of elementary 
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events 2 of process X by a mapping 7:94 Q. Then the 
expression 


1) = GO) F()) (3.12) 

gives the mapping of 9 in P=2X/:, We write 7X, off 
and o#*® for the elements defined by conditions 3,3.B-3.3.E. 
If we take into account the fact that elements © x,, of and 
ofS are connected with elements ‘. x, cMf, of* by expressions 
3,.1.4-3.1.D, we can deduce the following relationships between 
%, Xp, oj, oA and %, x, ofl, off * 

TIT (ol =F (0) ] 

STO ele) At SEE | (3.12) 

1 (o41) & off 

1 (8°) © of 


—_— 


We put for each A€ofS: 
P.2(A =P, T (AL (3.13) 


It may easily be show that all the conditions 2.1.A-2.1.F 
are satisfied for (x,t, o#, P,.2) . These elements thus define 
some Markov process ¥, Obviously, X is a canonical sub- 
process of process x, On combining (3.12)-(3.13) with con- 
ditions 2,11,A-2,11.D, we observe that the Markov process X 
is subordinate to process xX, 


3.4. In view of condition 3,.1.E’, for every subprocess of 
a Markov process X there is a corresponding system of func- 
tions of() (0<s<t, mE Q,) (functions 2% are measurable with 
respect to the s-algebra i). Here, every function a can 
be varied as desired over any set belonging to &; and having 
Ps, Measure zero for any x€E. We shall speak of two 
systems {%{} and {2%} as being equivalent if aj=2! (a,c. 

2, Ps.) for any O0<¢s<t,x€E. By lemma 3,2, there is a 
one-to-one correspondence between classes of equivalent sub- 
processes of a Markov process X and classes of equivalent 
systems of functions {a2} (if we consider only systems that 
correspond to some subprocess). Our main purpose is now to 
discover what sort of systems {a2} correspond to subprocesses 
of a Markov process X., The first step in this direction is 
provided by the following lemma, 


Lemma 3,3. Let X=(x, 7%, of}, P,») be a subprocess of the 
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Markov process X= (x, 6. Mt Ps,2) . The functions defined 
by condition 3,1.E (or 3.1.E’), possess the following proper- 
ties: 


3.4.4. For any s<t<u, x€E 


of, = afar, (a,c. Q,, Ps,x)- 
3.4.B. For any s<t, x€E 


Oc<a<i (a.c. Q,, Ps, x): 
3.4.0. For any s<t, x€E 


lim a; =27 (a,c, 2, Py). 
tye ® 


Proof. Property 3.4.B follows from 1.6.3), and property 
3.4.0 from 1.6.D, It remains to prove 3.4.4. 


We shall show that, if AGW? BEN , then 

B, (AB. & > uj} =M, [Zante]: (3.14) 

We note first of all that 
(B.C >aje dM. (3.15) 

where of, is the s-algebra in space g. generated by sets 
{x,€T, C>u} @Elt. u].TE#) . For the class of all subsets 
B of set &, that satisfy (3.15) is a c-algebra in @.. By 
(3.6), this c-algebra contains all sets {x,€V, >a} (vE le, a}, 
Tég) . It thus contains Wi. 


Since {A,C>1}€ off, {B,C> u}EoVu , we have on applying (2.9) 
to the process ¥: 


B,. {AB, t> a} = Pe {A, t>é, B, o> u) = (3,16) 
=M,,« (Xat>ePsz, (B o> 4)}. 
Moreover, by (3.3), 


P,y(B, C> u)=M, , (zp). (3.17) 
We have from (3.4), (3.16) and (3.17), 
P,,.(AB, C> w= M, . [2b.4My,», (0%) (3.18) 


On the other hand, we find on applying (2.10) to the 
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process X3: 
My, 2 [Xa%L5%] = My,» [2428M,, em (45%,)]- (3.19) 
On comparing (3.19) and (3.18), we get (3.14), 
Let # be the class of events Dca@, , for which 
B, (0. 0> w= M, , [zpatet). (3.20) 
and let @ be the class of all events AB, where AE€Wit, 
Be Wt . We have already shom that ¥>D@. It is clear 
that @ is a z-system and ¥ a i-system in @,, and by lemma 
1.1, # 2s(@). But obviously, o(@)=WN%, Hence F DW, 
Since (3.20) holds for any D€dWf% and since function ata’ is 
clearly o,-measurable, we have 
Pao (t>afWa}=ale, (a,c, QP) (3,21) 
On the other hand, by (3.1), 
P,2{C>ufMi}=o, (2.0.2, Py.) (3.22) 


Expressions (3,21) and (3.22) yield 3.4.4, and the lemma 
is proved, 


‘ 3.5. We now take the equivalent systems {ae} connected with 
a@ given subprocess X of process X by condition 3,.1,E’ and 
attempt to distinguish the one for which conditions 3.4.A- 
3.4.0 are satisfied in a stronger form (not for almost all, 
but for all ), 


We shall speak of the system of functions %(w)(0<s<t, 
w€2,) as defining a mltiplicative functional a of the 
Markov process X—(x;, (, off, Ps,2) if the at(w) satisfy the 
following conditions. 


3.5.4, at(w) is &-measurable, 

3.5.B. a8(w) at (w)=ak(w) (s<t<u, w€Q,). 
3.5.0. O<at(W)<1 (s<t, w€Q). 
3.5.D. ie © = (0) (s<t, w€ Q). 


We shall say that the subprocess X of Markov process X 
corresponds to the miltiplicative functional « if, for any 


62 THEORY OF MARKOY PROCESSES 


s<t, x€E 
Pe ale> thot} ai)  (a.c. 2p Pe, 2)- (3,23) 
Theorem 3,2, The sufficient condition for the subprocess 
X== (Xp 0, .oM§, Pez) Of Markov process X to correspond to 


some miltiplicative functional is that either of the follow- 
ing two conditions be satisfied: 


3.5.2. Ps, x; s, E)=P,,0@>s)=1 for any s>0, x€E. 


3.5.8. The functions of defined by condition 3,1.E° are 
subject to the convergence 
lim ofp of = (2.0. Qy Pe, 2) 
Bn fs 


for any s<t, x€E. 
Proof, a) If B,,{@>s}—=1, then by 1.6.3, , 


a= Poa {t>sfot}=1 (a.c. 2, Psa) (3.24) 
Let s,Js. By 3.4.4, 3.4.¢ and (3.24), ye have for almost 
all w€2, (in the sense of P,,2 % =A, 0%, Os —> ag 


and 81, Condition 3,5. follows from this. 


b) We now show that, as a consequence of 3,5.8, the sub- 
process corresponds to some multiplicative functional «, We 
shall start from any system of functions «af defined by con- 
dition 3,1,E° and construct an equivalent system of functions 
yielding the multiplicative functional «a, 


The set 
A= {af < 0} U {af > 1} 


belongs to Hi and, by 3.4.B, Ps,o(Ai)=0. We replace the 
values of functions af(w) on the set A? by zero, We get the 
equivalent system of functions which satisfies condition 
3.5.C. (Naturally, it also satisfies condition 3,1,E° and 
conditions 3.4,4-3.4.C) Our initial functions will be denoted 
by @ again, 


We put 
Qe u= (ats)  (s<t<u). 
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Obviously, Qs,4.ue NM . By (2.9), for any I<s we have 
Pave (Qs, 1,4) = f®,, vs (Qs, +, u) Po, x (dw). 
a5 
In view of 3.4.4, Ps,y(Qs,+,.)=0 for any y, and therefore 
Pox (Qs, 1,4) =9 for any x. 


Let 47(w) denote the exact upper bound of rational numbers 
such that ©€Qs,,. for certain rational u>t>s, Clearly, 
1()<S(#) and 


(summation over rational s,f,4), Hence it follows that, 
for any «CE, 


P,2{1>9}=0. (3.25) 


Condition 3,5.B is evidently satisfied for any rationalu>t>s 
of (4.0). Hence for any »€Q, and any rational f€(7,0),28(w) 

is a non-decreasing function of s (if $s runs through rational 
numbers of the interval (y(+). 4), Thus the limit exists 
for any real s>y(w): 


ato Chasinen@) (p rational ). (3.26) 
It is clear that 
o740 == a8 Had" (n(w) <s<q' <q<t(w), g and y rational )- (3.27) 


Hence 3+ is a non-increasing function of ¢ (for rational 
values of ‘f€(s,0)). We put 


aig= lima? @ rational. 1<s<t<Q). (3.28) 
If s<t<u are any real numbers from [y, t), we obtain by 
putting first g’ | ¢ then 9|s in (3,27): 


S+0 —— 98+0,t+0 
m0 at 0% 444.0" 


Let s<t<C(w). We put 


ast} (w), if yw<s, 


00) =| 0, if 4(w)>s. 
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These functions may easily be seen to satisfy conditions 
3,5.B-3.5.D, Furthermore, 


{a¢ = af} SC,uc., 
where 
C\=(1>s}, C= {n<s, aft? + ag}. 
By (3,25), for any x€E, 
Ps, 2 (C,) = 0. 


By (3.28) and (3.26), 


a= Q,, lim aP + a8 i : 
7 {U[ pve # a] u {limag + ap} 
(p-<q are rational numbers), By 3.5.8 and 3.4.C, we now 
conclude that p, ,(C.)=0 and therefore 


P,, , {a % af} = 0. (3.29) 


Function 2? is clearly oW"-measurable, In view of (3.29) 
it is &{-measurable and satisfies relationship (3.23). 


The theorem is thus proved, 


3.6. Theorem 3.3, For every multiplicative functional «z 
of the Markov process X=(x; & ofj, Py.) there is a corres- 
ponding subprocess X of X, 


Proof, It follows from theorem 3,1 that, if one subprocess 
corresponds to the functional «, there is a canunical subpro- 
cess corresponding to a, We shall therefore construct sub- 
process X in the canonical form, We define ©,7, x;,o#; and 
oM* by means of conditions 3,3,.A-3.3.B, It remains to assign 
the measures P, ,. 


We fix w€Q and s>0 and consider the function 
. for 0<t<s, 


Fe(t)={ 1—ai(w) for s<t<f(w), 
1 for #22 (). 
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In view of 3.5.B-3.5.D this function satisfies conditions 
1,11,4-1,11,C0, and in accordance with sec, 1,11, there 
exists on the segment [0. co] the probability measure 2 
(TE #) *) such that of, = Fe(r) and therefore a, op = 4 for 
wEQ,. - 


Let A€o#* , Then A€oM*X g*, and in accordance with 
lemma 1.4, the set 


Ay = {h:(w, 4) € A} 
belongs to #*S # for every w€2 . We put 
a4 (0) = a4, (0), (3.30) 
Obviously, for A=[C,C>#]X(f, co] (CE of’, t>s) 
aa(o)=zew)a(o) (EQ). (3.31) 


Hence we easily deduce by conventional use of lemma 1,2 
that the function a4(w) is of#*-measurable for any A€of*, We 
put 


P, (A) read Ma, td 3 232) 
Clearly, for every A€of#s, 
Ps (AX )= Py, 2 (A). 
It follows from (3,31) that 


P,(C.C>i} =P, n {IC C> AX, col} = 


= My axoet (CEM tas, (9-33) 


The system of elements (Xp & off, P,,.) evidently satisfies 
conditions 3,1,4-3.1.D. It follows from expression (3,33) 
that 


Pes CU >tfo#"} = at (@.c, 2p Ps, a) 


so that condition 3,1,E° and relationship (3,23) are satisfied, 


We also want to show that elements (Ken t, off, B,, 2) satisfy 
conditions 2,1,A-2,1.F, It is obvious that 2,1,A, 2.1.B and 


*)#* denotes the o-algebra in the space [s,+o] generated by 


all intervals, 
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2,1.C hold, It follows from (3,33) that P(s. x; t,T)=M, ,0f7, (x). 
Since the function 7%p(~,)a: is .s-measurable, it readily 
follows from lemma 2,2 that P(s, x; t,T) is a #-measurable 
function of x, so that condition 2,1,D is fulfilled, I+ is 
obvious that 2,1,E holds, It remains to prove condition 
2,1.F. 


We can more conveniently prove this last condition in the 
form 2.1.F’*, In accordance with 3.3.D, every set A of off§ 
has the form A=C X(t, co] (CE oM%) . By (3.33), 


B,.(A. %, ET) =P, o{IC, x, ET] X (a, ool} = 
= Mg, a l%okp (Xu) tl (3.34) 
We have on the basis of 3,5.B and 1.6.H, 


Me, x Xo%p (%y) a) = Ms, x [tkecp (*,) a1 =. 
= Mg, - (7co{Ms,x (AL Xp (uf Af]. 


(3.35) 


The function aiy,(*,)is \‘-measurable, We therefore have, 
on applying theorem 2.1 to the process X: 
Mg, 2 [Zp (x,)| As] = M;,, [al 7p (x1 = 

=P(t, x; 4, P) (a.c, 2, Py, .). 
We have from (3,34), (3.35) and (3,36): 

Py o(A, Xu€ T)=Me, « {xasP (t, x 4,T)}. (3 37) 


It may easily be seen from (3,33) that (3.4) holds, 
Relationship (3.37) can therefore be rewritten as 


(3.36) 


Pea (Ar HET) = Me, 2 (icles Pb xi w T= 
=M,2lxaPt, %5 4 TY) 
and condition 2,1.F° is satisfied, 


The theorem is now proved, 


3.7. We can connect with every multiplicative functional a 
of a process X a system of random variables &, which are 
defined as follows: 


inf (t:2¢(m)=0} (wE€Q,), 


= (3.38) 
Ss (w € &,) 


By (w) = { 
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(if of > 0 for all ¢€ [s, 0), we put § (o)=C). In view of 
3.5.D, the lower bound is attained in (3,38), so that af —0. 


Some obvious properties of the functions 5, may be noted, 
3.7 A, S<hs(0) <2 (w). 

3.7.B. (fs >t}ER: (OCs<d). 

3.7.0. bs >to SEF) Ocs cd. 


Lemma 3,4, If X is any subprocess of process X corres- 
ponding to the miltiplicative functional 2, we have for all 


s>0, x€E 3 
Ps, » {? <i} =i. (3 239) 
Proof, By 3.7.B, 1.6.F, (3.2), (3.23) and (3,38), 


Be, (>t > tefot’}| =\,5_ Ps, of S> thom} =0 6 
(aioe, Baa)s (3.40) 


Consequently 
2(t> t>t,| =0. (3 41) 


Let A be a denumerable set everywhere dense in the interval 
[s, 00). Then 


R>al= Ule>r>s). (3.42) 


Co (3.39) follows from a comparison of (3.41) and 
3.42). 


We shall consider in detail the important particular case 
when the functions a defining the multiplicative functional 
a take only two values: zero and unity, In this case the 
functional « is expressed in terms of functions § by 


a () = %_ 5 1 (@)- (3.43) 


If {&,} is any system of functions satisfying conditions 
3.7.A-3.7.C, the functions %—=X:,>¢ will satisfy require- 
ments 3.5 yee 5.D. Formlae (3, 38) and (3,43) thus define a 
one-to-one correspondence between multiplicative functionals 
with values zero and unity and all the possible systems of 
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functions §& subject to conditions 3.7.A-3.7.0. 


A comparison of (3,5) and (3,43) shows that the transition 
function of the subprocess is expressed in terms of § by 


Bis, x; 4, I= Py, 2 {ET & >t}. (3.44) 


Lemma 3,4 can be strengthened as follows in the case in 
question, 


Lemma 3,5. if X is any subprocess of process xX corres- 
ponding to the system {§,}, we have for all s<U,x€E 


Bielke>s. THE} =O (3.45) 
and for all x€E 
Bo, 2 (0% bo) =0. (3.46) 
Proof. For any f>s, 
Py a (CX t<E, fof} =LrcePs,2(b< th of?} = 


= Ieee, fl > :]=0 (2.¢ 2, P,, ,). 


(3.47) 


If A is a denumerable, everywhere dense subset of the 
interval [s, co) , we have 


(>s, C< Bl =U <1 <). (3.48) 


tEA 
It follows from (3.47) and (3,48) that 
Py a {be > S.C <E} =O, 
and we get (3,45) on taking into account (3.39). 


If s=0, the set {&<s,C<t,}=—(%=—0,%<0! is empty, and 
equation (3.46) therefore holds, 


2, Subprocesses Corresponding to Admissible Subsets. 
The Formation of Parts of a Process 


3.8. Let T be a subset of the phase space (F, ¢). We put *) 


*)Writing x, (w)er indicates that the value of x;() is either 


not defined or else belongs to EN\I , 
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Se (I) = §, (I, wo) = inf (t: > s, x,(w) ET} (wE Q) ( ) 
K=O) }. G49 

Let t<C(w), The segment of trajectory *, ¢<4uz<fis con- 
tained in T if ¢<£, (1), and is not contained in Tf if 
t>&(M) . It is therefore natural to speak of §(T) as the 
instant of first departure after s of the trajectory from 
set [, The functionst,—&,(l)always satisfy conditions 
3.7.4 and 3.7.C, If they also satisfy condition 3.7.3, we 
shall describe the set [ as admissible, Some subprocess of 
process X corresponds to each admissible set, 


The question as to whether a given set is admissible for a 
process X has to be considered on its merits in each case, 


Lemma 3,6. The sufficient conditions for a set I to he 
admissible for a process X are thata 


3.84, Y= ne {x,€T} CR. 
3.8.B, (&(@)>¢} Est and 


Pt, x; t, EY=P,, 2 {&@) >t}=1 for all t>0, xe€P. 


The transfer function of the subprocess corresponding to 
set I: is given by 


Ps, x; t, G)= Py, (Vi. %,€G}. (3.50) 
Proof, We have 
{8r >t} = (VE & 2) > ¢}. 


Hence by theorem 2,1, 


Pe, x {E, > t} = f Py, x, (5, (.) > t} P,, 2 (dw) = P,, 23 (Ts). 


oS 
¥y 


Py, o (WEN) >] =0 (3.51) 


ea (—,(7) > #}€ R?. Formula (3,50) follows from (3.44) and 


5 ° 


We note that, if process X is continuous, ~:,(r,o) (w) belongs 
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to the boundary of I*) for every w satisfying the inequality 
s<& (0) <C(w). If G is an open set, the boundary of G does 
not intersect with G and therefore 


{0:8 (G, 0) <C(w)} = [w: 6,0) EC ENG}. (3.52) 


Let 7 be an arbitrary o-function, It is natural to speak 
of §,()=£,q (0. ») as the first instant of departure after y 
from set [, ‘The reader will easily prove that 


EC) >n>s} S (8, = 8, OM}. (3.53) 


If G&L ,we have s<%,(G)<i,(r)*, and it is clear from 
(3.53) that 


TM) =a) WE). (3.54) 


The question as to whether set I is admissible is closely 
bound up with the question of the measurability of events 
wit) = {segment of trajectory x,(s<u<t) is contained in [}, 


If process X is given in a topological measurable space, 
it is important for us to investigate, in addition to events 
wWi(T) , the events: Wi (1) = {closure of segment of trajectory 
Xy(s< wee is contained in [}, 


A general investigation of the conditions of measurability 
of events Wim) and Wi(P) requires the use of the theory of 
continuation of capacities, The subject is therefore treated 
separately in a special supplement. 


3.9. When we construct a subprocess of a Markov process to 
correspond to an admissible set F » the phase space remains 
unchanged, This naturally suggests a modification to the 
construction so that the phase space is shortened to the set 
E. 


Let X= (xp 6 of, Ps, x) be a Markov process in the measurable 
space (EF, #)and let EC and. = : #(E}. We shall describe 
the Markov process X= (kp C Als, Py, 2) in the phase space 
é, _#) as part of process X if there’ exists a mapping 
1: @-+2 (Gand .Q are the spaces of elementary events for x 
and X respectively) such that: 


*)The boundary of a set ris defined as the intersection of 


the closure of lr with the closure of E\I, 
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3.9Ae TO =tE 1@)] WE 9). 
3.9.B. x)= x, [7 (0)] CES, 0<t< FO). 
3.9.0, If A€oM: , then {(77'A, C> 2} € ofl. 
3.9.D. of? D1*(oM*) and P,, g(q-1A)= Py, (A) for AC oft, x CE. 
The transition function of process X is given by 
P(s, x; LT)=Po, a (MET. be >t} (KEE 0<s<, (3.55) 


Theorem 3,4. A part of process xX can be generated on 
every non-accessible set satisfying condition 3,8.B, 


Proof, Since 
Wi (x, EE) € Ri, 


the set E satisfies condition 3.8.A, It is admissible by 
lemma 3,6. We now consider the subprocess X= (x, f, off}, P,, 2) 
of process X corresponding to the admissible set =, We put 


£=',(B, t=t(, S=C=t), F@)=o EB). 
Since E is non-accessible, we have 
{be > s} S {E=&}. 

Thus 

D(C & > 5) = (FE, > s}. (3.56) 

Let A€o# and A 2S. We now have, in view of (3.56), 
ADl=t,>s}=(&>s}\C#e b>} 

and, by 3.8.B and (3.45), Ps. 2(A)=1 for every x€E, Hence 
the * probability measures P,, 2(x€£) defined in the c-algebra 
oMe induce the probability measures P,, a2 in the o-algebra 
oht=7-10Me (in space §) (cf, the proof of theorem 2.5). 
We put 


hi = 7 of, TO) =tH Oh HO) =H OL 


The elements (Xp T, ofl, P,, 2) obviously form a Markov process 
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in space (£,%), This process is readily seen to be a part 
of process X, 


The theorem is thus proved, 


3.10. If E is any admissible set for process X and if X¥ 
is any subprocess corresponding to _—, there exists 3 by virtue 
of (3.44) and lemma 2,4 a process X equivalent to X for 
which the set F is inaccessible (lemma 2.4 is easily rewrit- 
ten for cut-off processes), Suppose that F satisfies condi- 
tion 3.8.B. Then by theorem 3.4 we can form a part X of 
process X on F, Process X will not in general be a part 
of process xX, however, since the passage from X to the 
equivalent X "will generally involve a widening of the space 
of elementary events (and the set of trajectories) and a 
narrowing of the basic c-algebras of, of’ , In order to 
generate a part of process X on the set & 3 certain restric- 
tions need to be imposed on X, viz we must require that its 
set of elementary events be not too narrow, or its basic s- 
algebras too wide, 


Theorem 3,5, Let the Markov process X=(x, C, off, P,,,) in 
the measurable space E€ # and the set(E, #)satisfy condi- 
tions 3.8,A and 3,8,B and the following conditions: 


3.10.A. offs dls. 


3.10.B, If x,()€E , there exists an w’ such that x,(w’)€E 
for all 2€[0, s] and ((w’) =C(w), x,(w)=x,(w) for all u€[s, C(w). 


A part of process X can now be generated on set E ‘ 


Proof, The set £ is admissible by reason of lemma 3,6, 
Let X=(x, ©, off, B,, z) be the canonical. subprocess of X 
corresponding to —, As in the Proof of theorem 3,4, it 
amounts to showing that the set S= (t=) satisfies the con- 
dition: If A€ of and Are, then Pp, .2(A=1 for all x¢€E, 
In our present case, T=2KX1 t= min Q, C), MS of X Ge 
NOX HF. 


We put w€A, if (@, NEA, By lemma.l.,4, 4,E€N¥, The 
inclusion 458 is equivalent to the system of inclusions 


A, 2{mind O='i} QED. (3.56") 


We put C€¥ if C possesses the following property: as 
soon as w€C, C(w)=C(w’) and x,(w) = x,(w’) for all uE[s, 6), 
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then »’€C, Evidently ¥ contains all sets {x,€T}(t>s, 
T€ #) and is a c-algebra, Therefore ¥ D\:, 


Let 4>S and let &(w)=£,(£, o)=min[, C(w)]. By virtue of 
condition 3.10.B, an w’ exists such that ¢(w’)={(w), x, (w’) 
=<,() for all u€Is.f(w)) and x,W@)€F for all x2€[0,s}. 
Bvidently &w)=tE, o)=8(E o)=minp (0). BY (3.56°) ,0’€ 4, 
nae since A,€o* we have w€A,. We have shown that, for all 

>S4 


A, 2 {i= min fi, C(w)}}, (3.57) 


It follows from the system of inclusions (3,57) that 


Therefore 


Pe, 2 (A)>Py, a (C= hs > St =P, 2 fe > 8} — 
[by 3.8.B and (3.45)]. 


3. Subprocesses Corresponding to Admissible 
Systems of Subsets 


8.11. Let F be a system of subsets of the measurable space 
(E, #). We put 


te(F) =F (F. 0) —=suphCs) — (WE), 
&(F ) = ko (oF ) = sup § (1). (3.58) 
rEg 


Let Gz denote the sum of all sets of system ¥, It is 
easily seen that 


&, (oF) <', (Gz). (3.59) 


We shall call §(#) the first instant of departure after s 
of a trajectory from the system of sets ¥, 


The functions §(%, w) always satisfy conditions 3.7.4 and 
3.7.0. If they also satisfy condition 3.7.3, we shall say 
that system ¥ is admissible, Im accordance with sec, 3.7, 


T4 THEORY OF MARKOY PROCESSES 


for every admissible system SF there is a corresponding class 
of equivalent subprocesses X of process X for which 


B, (= 8(F)} = 1. (3.60) 
We shall describe system ¥ as subordinate to system ¥ ‘if 
there exists for every setT€¥ a set IY€ ¥ containing it. 
It is evident from this that &,(%)<&(¥’)for any s>o0. It 
may be remarked that all subsystems of # are subordinate to 
F. We shall refer to ¥ and ¥'as equivalent systems if # 
is subordinate to ¥’ and ¥ subordinate to ¥, In this case, 
5,(¥)='.(¥’) for any s, so that equivalent systems define 
the same class of subprocesses of X, 


On replacing s in §(%, w) by any w-function 7(w), we get 
a quantity §,(#) which it is natural to call the first 
instant of departure after 7 of the trajectory from system 


&F. We have from (3,53) and (3,58): 


= 
wor 
=~ 


F)>y=—Un=s. (F)>sl ES 


EUnas KM=EM=KN=4H).  F-) 


s 


3.12. We shall describe the system of subsets ¥ of a topo- 
logical measurable space (£, @. #) as normal if an equivalent 


system [,,...1T,,, ..., can be found satisfying the following 
conditions: 

3,12.A, T,..., Ty... CF. 

3.12.8, T,,..., T,, ... are closed, 


3.12.C, For any n there exists G,€ @ such that T,c¢G, 
ST n+. 


We observe that 
= limsé 
bs (F ) ee (Pn). G . 62) 


Some important examples of normal systems will now be dis- 
cussed, 


3.12.1, Let (E. p, #) be a metric measurable space and let 
F be the class of all bounded sets of this space, We 
choose any point 2 in E and put - T,={x:p(x, a)<n} . The 
sequence {f,} is obviously equivalent to system ¥ and satis- 
fies requirement 3,12.B, We see on putting Ga={xip(x, a) 
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<n +3} that it also satisfies requirement 3,12.C, We can 
satisfy requirement 3,12,4 simply by supposing that # > @or 
that p(x, a) is a #-measurable function of x, 


3.12.2, let G be an open set in the metric measurable 
space (E, p, #) and let ¥ be the class of all sets f for 
which o(I, E\G)>0. The sequence T= {xi p(x ENO>— 


is equivalent to ¥ and satisfies conditions 3,12.B and 
3.12.0(/ x: p(x, ENG)> 5-5; }) can be taken as G,). If #2C, 


the continuity of the function p(x, E\G)(ef. sec, 1,8) 
implies that condition 3,12,A holds, 





3.12.3. Let the topological measurable space (F, @, g) 
have the following properties: 


3.12,3.A, For every point x a measurable bicompact set Q 
and a measurable open set.U can be found such that x€USQ,. 


3,12,3.B. A sequence of measurable bicompact subsets E,, 
exists such that ,_ UEn* *) 
=o n 


nal 


We shall write ¥ for the system of all measurable bicon- 
pact subsets of the space (E£, @. #). 


We note that, for any setTC¥n, there exist sets UE CN 
#8 and QCFf/#Fsuch that rouca. For, dy 3.12,3.A, 
for every x«€T there exist U(x)€ CN #and Q()EF/) ¥ such 
that x€U(x)SQ(x). Sets U(x) form a covering of T, anda 
finite covering U(x)... on U (%m) can be chosen from this, 
Sets u=Yue ) and e=U Q(%) obviously have the propert= 
1 


1 
ies that we require, 


We start from the system £, defined in condition 5,12,3.B 
and construct sequences [,, and G, by the following recurr- 
ence method, We put P,—£,, Assuming set [, to be already 
constructed, we choose sets G,€ C# and Q,.€F % such that 
T,£6,6Q, , and put Iy41=Q,UE,41 . Sequences {I} and {G,} 
obviously satisfy conditions 3,12,A-3,12.C, Sets G, more- 
over form a covering for any T€¥, and since a finite 


*) Property 3,12,3.4 is a strengthened type of local bicom- 


pactness, whilst 3.12.3.B is a strengthened type of s-bicom- 
pactness (cf. sec, 1.7). 
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covering can be chosen from this, and G,©]...€6,©..., we 
have TeG,S7,., for any an, It is clear from this that (I,} 
is equivalent to ¥., 


3.12.4, Let G be a subset of the topological measurable 
space (F, @, g) satisfying the following conditions: 


3,12.4.4, For every x€G a measurable open set UEG and a 
measurable bicompact set QeG can be found such thatxE€Gcaq, 


3.12.4.B, G= UE, » Where the £, are measurable bicom- 
pact sets, ntl 


On repeating the arguments of example 3,12,3, the system 
of all measurable bicompact subsets of set G will be seen to 
be normal, 


Lemma 3,7. let # be a normal system of subsets of a topo- 
logical measurable space (£,@.9). NowGzs€#f @. The con- 
verse holds if space (FE. @, #)is subject to requirement 1.9.B: 
every set GE #()@ can be written in the form G=G,, where 
& is a normal system, 


Proof, The first assertion is obvious, To prove the sec- 
ond, we consider a measurable continuous function f such that 


G={x: f(x) > 0}, 
and put 


T= {#:f@)> =} (t=1, 2,...). 


It is easily seen that ¥ wiley is a normal system that satis- 
fies the condition Gz=G (we can take : 2 as 
Gn). {f@)>s24} 


Lemma 3,8. If the system ¥ consists of compact sets, it 
is subordinate to any normal system ¥ satisfying the condi- 
tion GzeG,. 


Proof, let I, be a sequence equivalent to # and subject 
to conditions 3,12,A-3,12,C, and let G, be the sets described 
in condition 3,12,.C _ The system {G,} is a denumerable cover- 
ing for any set TE. Hence a number 2 can be found such 
that ToG,CT,41. System ¥ is thus subordinate to the 
sequence {I} and therefore also subordinate to ¥, 
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Corollary. LetG€#f\C. All the normal systems ¥ con- 
sisting of compact sets and satisfying the condition Gz=—@G 
are equivalent to each other, 


3.13. Let X. be a Markov process in the topological measur- 
able space (F, @, #), where the space satisfies 1,9,B, and 
letGE€ CN#. In accordance with lemma 3.7, @ normal system 
& can be constructed for which Gz,—G. We agree to write 


%(G) =§,.F) (3.63) 


and to call *,(G) the first instant after s of departure of 
the trajectory from the interior of the open set G. The 
value of t,(G) is dependent in general on the choice of system 
& as well as on G. As we have just seen, however, all sys- 
tems ¥ consisting of compact sets lead to the same value of 
1,(G). We shall see later (cf. lemma 3,10) that the proviso. 
regarding compactness..ceases. to have any importance when the 
process X is continuous, 


We shall write «(G) instead of %)(G) and +,(G) instead of 
t,(F) (G=Gz) - 


Lemma 3,9. let X be a Markov process continuous from the 
right in the topological measurable space (EF, @, g). If 
U, GE Cf\# and the closure of U is compact and contained in 
G, we have 


t(7) (G) = +, (G). (3.64) 


Proof, It follows from (3.61) that, for any function 7(w) 
(wE Q,) » 
{%4(G) >1>s} & (%4(G)=+,()}. (3.65) 


Let # =({T,} be a sequence satisfying conditions 5,12.A- 
3.12.C and such that Gz=G. Let G, be the sets defined 
by condition 3.12.C. The same method as used in proving 
lemma 3,8 shows that, for a certain m , the set G,, contains 
the closure of U. Hence it follows that UcI, for all 
n>m. We note that x,€U for s<t<1,(U). If mE, 
we have ~,€G, for some n>», and, in view of the fact that 
the process is continuous from the right, there exists «>0 
such that x,€G, STn+i for alls<t<+,(U)+e2. Consequently 


{mE Gls {t.(G) > =, (U)}. (3.66) 
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On putting 4=+,(U) in (3.65) and comparing with (3,66), 
we get 


(£2, (7) € G} S {ts (G) = % xy (G)}- (3.67) 


Expression (3,64) follows from (3,67) and the obvious 
inclusion 


(90 € G} S [44 (G)=44(U), te) (C) = 4, (G)). 


3.14, Lemma 3,10. A normal system of subsets F of a topo- 
logical measurable space (E, @, #) is admissible for every 
Markov process X continuous from the right, If xX is con- 
tinuous, we have 


i(F)=t(Gr) (s>0). (3.68) 


Proof. a) We consider the sequence ({I,}, equivalent to ¥ 
and satisfying conditions 3,12,A-3,12.C, We have from (3.62): 


foe) 


{6s (F) > t} = Uk Pn) > ¢}- (3.69) 


n=l 


In accordance with sec, 3.8, we put 
WiM= f) {x.€T}. 
uwEls t] 

If wE€W;(,), and consequently x,(w)E€P, for “Els, 4], the 
fact that X is continuous from the right means that a 6>0 
exists such that x,(w)€G, for 4€ls, ¢+2), Hence & (41,0) >t, 
Therefore Wi(P,) ¢ (&,(ny1) >t}. It follows from the inclusion 

ta {E5 Ga+) > the Aa (Tn+1) 
and forma (3.69) that 


i (F) >= Vy. 
{Es ( ) J U t(P,) (3.70) 


‘If A is any denumerable rverywhere dense subset of the seg- 
ment [s, 4] containing the point t, we have 


WEP.) =) (xu€ Pr} € NE. (Gz) 


Uuea 


It follows from (3.70) and (3.71) that {&(#)>r}EN? . 
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The first assertion of the lemma is thus proved, 


b) In view of (3.59), we can prove (3,68) simply by show- 
ing that 


&. (oF) >§, (0). (3.72) 


This inequality is trivial in the case when §,(¥, 0) > 
Cwm). If &(F. 0) <C(w), it follows from the fact that now 
%,(7)EG . We show that 


(Es (F) <C}S {xe (57) EG}. 


On putting t=s in (3.70), we get 


oo fo) 
3(F) > s}= Y Yin) = U (x €T,} = (x, € 6). 

nal n=1 
It is clear from this that [:(7)=s}S{*, 5,€G], It 
remains to consider the case when gs ct (F,w)<C(w). BY 
(3.62), starting from a certain n we have 5 <i, (P,)<C(w). 
It now follows, as shown in sec, 3,8, that ~:,(r,) belongs to 
the boundary of I, and, in view of 3,12.C, *,(r)€@, for 
m<n. On letting n--oco and taking into account the fact 
that Pile 00 aaa (#), we conclude that x, .5,éG,, for any 
m, 80 that x, (9G. . 

8 


Corollary. Let xX be a continuous Markov process in the 
topological measurable space (£, @, #), where the space 
satisfies condition 1.9.B, Every set GE@CN #is now 
admissible for process X, 


For, in view of lemma 3.7, there exists a normal system ¥ 
such that G=Gs. By lemma 3,10, this system is admissible 
and & (¥)=§ (Gz). Therefore 


{(G) >t} = [6 (F) > t} CRs, 
which is what we wished to prove. 


3.15. Let X=(x;, 6, M3, P.,2) be a Markov process in the 
measurable space (£, #), «¢ a system of subsets of the space 
2 ,G=Gyand #=[G]. We shall describe the Markov pro- 
cess X=(Xx, ©, oi, Ps) of space (G. #) as the part of process 
X corresponding to the system # if there exists a mapping 
7:9+@ (& and @ are spaces of elementary events for 
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X and X) such that: 
3.15.4. Co) =F, 7(@)] @E®). 
3.15.B. x (o) = %[1(@)] @ER, 0<t <&o)). 
3,15.C. If A€oMi, then {77'A, T>t} CoM. 
3,15.D. of’ 2 7"(o4’) and B,.2(17'A)=P.,0(A) for AC of, x€G. 


When §(¥%)=—{(Gy), this definition becomes the definition 
of sec, 3.9 for a part of process X on the set Gz. 


The following theorem is based on lemma 3,10 and is got by 
repeating almost word for word the proof of theorem 3.5, 


Theorem 3,6, let F be a normal system of subsets of the 
topological measurable space (£. @, #), where all sets of F 
satisfy condition 3.10.B, For every process X continuous 
from the right in the space (F, @. #) and subject to condi- 
tion 3.10,A, a part can be generated corresponding to the 
system ¥, 


4, The Integral Type of Multiplicative Functional 
and _ the Corresponding Subprocesses 


8.16. Theorem 3,7. Let the Markov process ¥=(x, 6. of, Ps, 2) 
in the measurable space (E, #) satisfy the condition: 


3.16.4. For any 0<s<#, the mapping defined by the func- 
tion x,(w) of (x 2.9% x Ki)in (E. #)is measurable, 


Let V(u, x) (4 >0, x € E) be a non-negative #9, X #-measur- 
able function and » a measure on the c-algebra #2. The 
expression 


af(w)==exp|— | V(u, x,)p (du Q 
: P| a (a, 4) 2 ] @EM (5 93) 
now defines a multiplicative functional of process X, pro- 
vided only that the integral in the expression is convergent 
for any 0c¢s<ct<Cw)- 


Proof, The mapping defined by the function V (4, x,(w))of the 
space (I; x 2,97 xX Ri)into (i, g%.) can be written as the pro- 
duct Va, where « is the mapping of (x2, #3 &%) into 
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(EXE, #tX #) given by 


a(4, w)= {4, x,(w)}. 


In view of 3,16,A and lemma 1,3, the mapping 2 is measur- 
able, By hypothesis, the mapping V is also measurable, 
Thus Va is the measurable mapping of (/} x 2;, 9? X &}) in 
(Z, ®eo) 5 i.e. the functionV (x, x,(w)) (wef, 0€2,) is PIX R- 
measurable, It follows from this, by lemma 1.7, that the 
function fvw x,)p(du) is &-measurable, so that the same 

(sy t] 
can be said of the function ot, The system 2} thus satis- 
fies condition 3,5,A, It obviously also satisfies conditions 
3.5.B, 3.5.6 and 3.5.D, The theorem is thus proved, 


The functionals defined by (3.73) will be described as 
multiplicative functionals of the integral type. 


Let us consider the subprocess of process X corresponding 
to miltiplicative function (3.73), We have for this: 


a sth[ i} = 
=exp/— [Vo *e@9] (2.0. Q¢45 Py). (3474) 
(8. 3+h] 


Suppose that, for every wE€2,V(u, x,)is a function of u con- 
tinuous from the right in the interval [0,€(w)). It now 
follows from (3.74) that 


Py a{S>sthyh} =1—V(s, %,) p(s, S-LA][1 +e (A)] 
(@.C. Qe Pe x) 


where e(h)-+0 8S AIO. 


Hence, on the assumption that the course of the process is 
known after the instant s, the probability that it will be 
cut off after a time (s, s-+A} is equal to V(s, x,)u(s, s+A] 
to an accuracy of higher order infinitesimals, 


3.17. We now suppose that measure p coincides with the 
Lebesgue measure *) and investigate the connexion between 
the transition functions of process X and its subprocess 


*) The case when » is absolutely continuous with respect to 


the Lebesgue measure with a density of p(x) reduces to the 
case of Lebesgue measure by replacing V by PV, 
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corresponding to functional (3.73). 


Theorem 3,8, Let X == (x;, C, oft}, Ps, 2) be a Markov process 
satisfying condition 3,16,A, and let V(u, x) (u>0, x€ BE) be 
a non-negative*) #°, xX #-measurable function, We put 


t 
stt=e|— FV xa (EQ). (3.75) 


If the integral in (3.75) is convergent for any 0<s<t<C(w) 
the function P(s,x;t,T)=M, ,[o8%r(*,] is connected with the 
function P(s, x; t, T)=P,,.(%,€F) by the integral equation 


t 
Ps, x; t, n+l fre. x; u, dy)V (a, y) Pu, yt, Dda= (3.76) 
s E 
= Ps, rig t, fT). 
Proof. For any ¢>0 and any w€Q, fv @ro » and conse- 


quently af also, is an absolutely continuous function of Ss, 


Therefore 
i t 
a i— f Fbaa=i— fv(u, %,) oY du. 
8 & 


On multiplying both sides by Xr (*,) and passing to the mathe- 
matical expectations, we have 


Pos, x3, T= 


: (3.77) 
=P(s, x: t, n—Me[ SY rabie(x) ae]. 


*)The reader may observe on running through the proof that 


our proposition still holds when the function V(u, x) takes 
complex values of any kind, provided only that, for all «¢cE 
and 0<s<t 


Mg, 2 | #2 |<K,<00, 


t 
J] JPe 3 seniven ni aac ie<en 
E 
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By Fubini's theorem (see 1,.4,.C), the last term is equal to 


J Mg, o[V (4 *,) xp (*,)] du (3.78) 


In view of (2,10) and lemma 1,6: 
Ms, o[Y (% +.) tM (*))] = 
= M,, bas {Vv (4 ¥,) M,, x, [Yep (*)}} a=, 
=M,,o1V(m x) Pw, xy t N= 
=f Ps, x; a, dy)V(u, y) Pu, ys tt PD. 
B 


(3.79) 


A comparison of (3.77), (3.78) and (3.79) gives us (3.76). 


5. Stationary Subprocesses of Stationary 
Markov _ Processes 





3.18. Two variants of a theory of stationary subprocesses 
can be described, corresponding to the two meanings attached 
to the term "stationary Markov process" (cf, sec, 2.8). The 
first variant is discussed in secs, 3,18-3,20, the second in 
secs, 3,21-3.24, 


We start in the first variant from the idea of a stationary 
process as the Markov process (x;,, C, off, Ps.) for which a sys- 
tem of operators 6, exists, satisfying requirements 2,5,A- 
2.5.0, 


Let X=(Xx, C, of%, P,,2) be a stationary subprocess of the 
stationary Markov process y—(x;, £, off, P,,2) and let 6, 9, be 
the operators corresponding to xX, ¥ and satisfying condi- 
tions 2,5,A-2,5.C, It may easily be seen that we now have 
(cf, the derivation of expression (2.32) on p.51) 


8: (7-1, > 0) = (7-28,4, F > t} (3.80) 
for every AGW". 


On equating 7-14 to A and 7-'8,4 to 6A, we can write 
(3.80) in the following more compact form: 


6,{A, T> 0} = (8,4,T>t} (AEN). (3,81) 
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. Let X bea stationary subprocess of the 
stationary Markov process X and let a§ be functions defined 
by condition 3.1.E, Then for any O0<cs<t, x€E, 


0,3 =o a.c. 2,, P,, 2) (3.82) 
Conversely, if condition (3.82) is fulfilled, there are 
stationary subprocesses in the class of equivalent subproces- 
ses corresponding to the system of °%]; or more precisely, 


a canonical subprocess belonging to the class in question is 
stationary. 


Proof, a) Let ¥=(x;,%, of%, Pz) be a stationary subprocess 


of the stationary process X=(x, 6, off, Pz). Let BE. 
We have by 2.5.4, (3.81) and 2.6.C, 


y 


6, (B, T> t—s} =3, (8, T> 0)6, (F>t—s} = 
= (0,8, €> s} (T >t} = (0,8, C >t}. 


It follows from this, in accordance with 2.5,C, that 
B,, » (0,8. €>t}—= BP, .(B, @>t—s}. 
Using (3.3) » the last relationship can be rewritten as 
Mg, 2hs,5% = Mo, okat—s (3.83) 
We have on the basis of 2.6.F and 2.6.D, 
Mo, 2Xp%—— = M,, 29, [X2%t-5 =M,, aXe, 3°07 (3.84) 


Functions % and 9,07, are WN;-measurable, Since W}=6,N\?_,.¢ 
Ae'g S , it follows from (3.83) and (3.84) that 


i agp, (dw) = f 0,09 ,P,,, (dw) 
A A 
for any ACAN%. We easily get (3.82) from this. 
b) Let X=(X;, %, od, P.,2) be a canonical subprocess of X 
and let functions o! satisfy conditions 3.1.E and (3.82). 


We observe that 


NS NX BHO, +00} 


where #0, +0) is the c-algebra in the space (0, +00], genera- 
ted by all intervals, Every set A€ajf’* can be written 
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uniquely in the form 


A= Ay XA, 
U n (3.85) 
where A,€of*, We put 
B= UY %AXxa+to. (3.86) 
O<Ag+0 


In particular, for any CEWN*,r>05 


8, {C, > A) =3, (IC, C> bh) X (, col} = (3.87) 
=[6,C, C>t+AlX +A, co] =(8,C.T>t+A}. 
The operators 6, are easily seen to satisfy conditions 
2,5.4-2.5.B, We shall prove that they also satisfy 2.5.C. 
If 
A=(C.T>h} (CEN, 2>0), (3.88) 


we have by (3.87), (3.3), 2.5.0 and (3,82), 
~ ~ ~ ~ x 
Ps, 2 (6:4) = Py, 2 (8,C, 0 > th] = My o%5 c= 
= My, 29: [Xc%] = Mo, 2 [Xc%] = (3.89) 
=P, 2 {C.t> hk} =P, 2 (A). 
The sets of type (3.88) form ax-system @. On the other 
hand, the class ¥ of all sets A for which condition 2,5.C 
is fulfilled is a \-system in the space 9,. In accordance 


with (3.89), %2@ By lemmal.l,  2:(@)=d, This proves 
the theoren, 


3.19. Theorem 3,10. Let Y=(x, 0 of, P.,2) be the stationary 
subprocess of the stationary process X=(x. 5 ofl, Py, x) 
corresponding to the multiplicative functional a= [az} . The 
random variables §, defined by (3.38) now satisfy the rela- 
tionship 


E,== s+ 8,6 (a,c, Q;, Ps, 2). (3.90) 


If §& is any system of random variables satisfying conditions 
3.7.4-3.7.C and (3.90), the canonical subprocess correspond- 
ing to the multiplicative functional 


= Xeoe (3.91) 
is a stationary subprocess, 
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Proof, We have for every r>0, 
(io <r} & {ar = 0} 
and therefore 
{Bbo <r} & {Oar = 0} & [a8,,—= 0} U (0,02 + o84,}. 
On the other hand, 
{(E.—s >r} = {8 > str} & {a8.,> 0}. 
Hence 
{Ea —s > r > OE} & (0,0? + a8} 

and by (3.82), 


Ps, 2 (fs —S > 7 > 0,8} = 0. (3.92) 


If A is the set of all non-negative rational numbers, it 
is obvious that 


{Fs —s > 8,6} = UJ {&, —s > > 8,85}. 


reA 
It thus follows from (3.92) that 


Ps, x {§s—s > O56} = 0. (3.93) 
It may be proved in the same way that 


Ps, a {fs —S < 84%} = 0. (3.94) 
We get (3.90) from (3.93) and (3.94). 


Moreover, if the quantities §& satisfy conditions 3.7.A- 
3.7.C, expression (3.91) defines, in accordance with sec, 
3.7, a multiplicative functional of process X. If (3.90) 
is satisfied in addition, we have 

0 
Set 1 = 8646 5 1g = 48, (> t-2) = X= a. 
(a.c. 2, Ps, 2), 
i.e. condition (3,82) is fulfilled, It follows from this, 


by theorem 3.9, that the canonical subprocess corresponding 
to functional {3 91) is stationary, 
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3.20. We shall consider some examples, 


3.20.1, Let T be a subset of the measurable space (E, #) 
admissible for the stationary Markov process x, and let the 
random variables §,=§,(I) be defined by expression (3.49). 
We have 


Hence 
{66>t}=U {%y.2€T)] = (> st. 


a 1 
0 = 
sustts 


The random variables §,==§,(I') thus satisfy condition (3,90) 
and, in accordance with theorem 3,10, we can in fact assume 
that the subprocess of X corresponding to the admissible set 
T is stationary, 


We leave it to the reader to show that the same situation 
applies for any part of a stationary process xX, 


3.20.2, Let a system ¥F of subsets of the phase space be 
admissible for a stationary Markov process xy. Condition 
(3.90) is easily shown to be fulfilled for the quantities 
defined by expression (3.58). A subprocess of X correspond- 
ing to the admissible systen F can therefore be chosen such 
that it is stationary. The same thing can be said as regards 
the part of a stationary process corresponding to some sys- 
tem of subsets F, 


3.20.3. We take a stationary Markov process X¥=(x,, %, of, 
Pz,2) satisfying condition 3,16,A, and let a= {a?}be the 
multiplicative functional of the integral type defined by 
forma (3.73). If » coincides with the Lebesgue measure, 
whilst the function V(uz, x) is independent of u, we have 


ais = Bate. (3.95) 
For suppose we put 


t-8 


n= f V@%dde 
Q 


and write L, for the set of all #4 -measurable p-summable 
functions in the interval [0, t—s] , for which 
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t-s 
f 9 (u)du=a. 
0 


We have 
f=3j=U fh Vevr=-~} 
and by 2.5.A, 2.6. and 2.6.6, 


Ga=a3= UY ff) Veud=9@)= 
e€Lg OSuKsi-s 


ts t (3.96) 
=f Vereddumal—=| fVeoaomal, 
0 8 
It follows from (3.96), in accordance with 2.6.B, that 
t 
bn f V(e,) ae. (3.97) 


We get (3.95) by comparing (3.73), (3.97) and 2.6.E, 


We conclude from relationship (3.95), on the basis of 
theorem 3,9, that the functional a corresponds to a station- 
ary subprocess of process xX, 


8.21. We now base our remarks on the concept of a stationary 
Markov process as a system(x;, C. of,, P,, 6,) satisfying conditions 
2,8,A-2.8.H, The theory of subprocesses admits &° number 
of important simplifications from this point of view, 


Let X= (xp 6 of, Pz, 6, and X= (x, off, B,,3,) be two station- 
ary Markov processes in the same phase space (E, g) and tet 
Q and & be the spaces of elementary events of X ana ¥ 
respectively, We shall speak of X as a subprocess of X if 
there exists a mapping ;: @-+2 such that: 


3.21.8, TO)<Ciy@)] @E8). 
3.21.3, %(0)=x,[7(o)] (0B, 0<¢ <%(9)). 
3.21.0, If A€o#,, then {y-14, >t} €ok,. 


~ 


3,21.D, of > 7-1940 and P,(;-1A) =P, (A) for AG of?, 
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3,21.E, P,{2>t| 1-104} =a[7(0)] (a.c. 7-12, B,). 
where a,(w) is anof’,-measurable function, 


On equating as usual? (7 (0) 1 (@€ 8) with 9(w) EQ) , 7-14 
with. £,17'F% =(1"'Ajwith F —{(A} , we can rewrite the condi- 
tions as follows: 


> 


3.21.8°, Cece. 

3,21,.B°, =x O<t<?). 

3.21.0°, If A€oM, ,then {A, T>z}Eoh,. 
3,21.D°, o&Do# and P,(A)=P,(A) for AC ofp. 


3.21.8, Pp (t>tlo#}—a, (a,c, P,), 
where a, is an of,-measurable function, 


If & denotes the class of all sets A€o# such that, for 
some BE ol", P,(A\ AB) = P,(B\ AB)=O0for all x€F, condition 
3,21,E can be put in the following form: 


3,.21.8°", PB, {C>tlo#t—2, (a.c,2,, P,) 
where a, is some &,-measurable function *), 


We remark that, if (x, 6 Hi, P,.) is a stationary subpro- 
cess of the stationary Markov process (x; [, of7, Ps, 2) (in the 
sense of sec, 3,18) and if the operators 0, §; are so chosen 
that conditions 2,°,A-2,5,.0 are fulfilled, (x,, 7, o#%, Bos, &) 
is a subprocess of the process (x; C, off,Po,2, 6,).. Is the con- 
verse true? Or more precisely, can we assert that, if xX-= 
(x, & o%,, P,, 9) is a subprocess of X=(x, 6 of, Px, 5), there 
exist stationary Markov processes %’ —(z,, 7, off, P,, ,) and 
X= (xp © off, Pec) such that X.,¥, x.,x and X is a 
stationary subprocess of x’ in the sense of sec, 3,187 This 
question can be answered in the affirmative at least if it 
is assumed that P,{?>0}—=1for all «€£, The proof is 
omitted since we shall make no use of the proposition, 

3.22. We shall describe the subprocess X=(x,, %, off, P,. 6,) » 
of process X= (x, C, of, P,, 8, a8 a canonical subprocess if: 


*)The whole of the theory still holds good if we put 


R= NNN, , where the definition of of,,,in terms 
of W; is the same as that of dt, in terms of of? (see foot- 
note on page 55), 
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3,22.A, The space of elementary events S&S of process X is 
equal to 2x], where 2 is the space of elementary events of 
process X and /=[0, oo]; the mapping 7: :8-.9 is given 
by 

1 )=o WER, AED. 


3.22.B, C(w, )=min[C(o), A. 
3,22.C, ¥4(0, N=-x,() for V<¢ <t<Fo, »). 


3,22.D. of,consists of all sets of the form AX(f, co] where 
A€oh, «+ 


3.22.8, of@ is the c-algebra in space § generated by sets 
{A, 0 >t}=[4,6 > t]X¢, oc] (AE oft > 0). 


We shall speak of the system of functions @() ¢>0,0€2,) 
as defining a multiplicative functional of Markov process 
x= (Xz, im A, P,, 6.) ifs 


3,22.a, % is &,-measurable, 

3.22.8, bet, =e (A.C. Qin Py) (8, t>0, x€E). 
3.22.7, 0S acl (¢>0, o€Q,). 

3.22.8, a %, (W) =a (0) ESO. o€ Q). 


Two stationary multiplicative functionals « and @ will be 
said to be equivalent if, for any ¢>0, x€E; 


Oy = (a.c. Q, P,). 


We shall say that the subprocess X=(x, t, &yP,, 8, of 
X= (x; G of, P,, 6,) corresponds to the mltiplicative func- 
tional a={a,} if functions a, satisfy condition 3.21.B’’, 


Theorem 3,11. A one-to-one correspondence holds between 
all the classes of equivalent subprocesses of a stationary 
Markov process X=(x; C04, P,, 6,) and all the classes of 
equivalent multiplicative functionals of x. Each class of 
equivalent subprocesses contains one and only one canonical 
process ¥ and consists of all the stationary processes sub- 
ordinate to X . The transition function of the process cor- 
responding to the functional a= {a,}is equal to 


P(t, x, T) = MoXp (*) (3.98) 
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Proof, a) It may be seen by repeating the arguments of 
lemma 3.3 with obvious modifications that, if X= (x, %, Ay 
P.. &) is a subprocess of the stationary process |X, the 
functions a, defined by condition 3,21.E satisfy the relation- 


ships 


0<ay<1 (2.0.2, P,), (3.99) 
1m, =% (a,c. Q,, P,) (3,100) 


together with relationship 3.22.8, We put 


Qpe a = (Ap 2 pteg # Mp4} U {Ap < O}U {ap > 1}U 
U {8 g0q <0} U (8,2, > 1}. 

It follows from properties 3,22,8 and (3.99) that P,(Qp, 
o=0 for any p,g>0 x€E. Let Q denote the sum of sets 
Qp,q Over all rational p and gq, Obviously, P,(Q)—0 for all 
xEE . 


If w¢€Q, for any rational p.q of [0,€(w)) we have the 
inequality 


0< wig) < Gp (w). 


Thus, for any ¢€ [0,%(w)), @ limit exists for % when p tends 
to ¢ taking rational values greater than t, We put, 


limap(o), if w&Q 
2%, (w) = [> 
1, if €Q. 


We have for any x€E, in view of (3.100), 
Pa {a 4) =0. (3,101) 


The function a,(w) is obviously W\-measurable; and by 
(3.101), it is &,-measurable. It follows also from (3,101) 
that functions z, a, Satisfy conditions 3,22,8 and 3,21, B?%, 
They evidently sles satisfy conditions 3.22.7 and 3.22.6, 


b) Let a={a,} be any multiplicative functional of process 
Xx - We shall show that there is a canonical subprocess X= 
(Xp & of, P,. 9)of X corresponding to a, We find §, & x, 
ot, o#oby means of conditions 3,22,A-3,22,E and associate 
with each w€@2 the probability measure 2,(w), on the o- 
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algebra #1, given by the requirement that 


={*o if t<C(w), 
lM =) 9. se EDCw) 


(the existence and uniqueness of this measure follows from 
properties 3.22.1 -3.22.a of the functions «a, combined with 
the results of sec, 1,11.), We put for each A€o# and each 
wEQ, 


Ay = {h: (w, 4) € A}, 
ay (o)=2,, (). 


B, (A) = M,2,. 


In particular, for any Coch, 
B,(C. T>t} =P, (IC. 6 > 11 XC, col} =Maxg% (3,102) 


We define the operators in by the same formulae as in the 
proof of theorem 3,9 [see (3: 85)- (3. 86)]. The reader will 
easily verify that the system (Sp a bn. P,, 4) satisfies con- 
ditions 2,8,A-2.8.H and 3,21,A-3,21.E, It therefore defines 
the subprocess of X corresponding to the functional a, 


c) The fact that (3.98) holds may be seen by repeating the 
argument of the proof of lemma 3,1, It follows from (3.98) 
that subprocesses corresponding to equivalent functionals are 
mutually equivalent, On the other hand, our arguments when 
proving lemma 3,2 show that, if two subprocesses of a process 
X are equivalent, they correspond to equivalent functionals, 
Finally, an almost word for word repetition of the proof of 
theorem 3,1 shows that each class of equivalent subprocesses 
of X contains one and only one canonical subprocess X and 
consist of all the subprocesses subordinate to X, 


The proof of the theorem is now complete, 


8.28. Theorem 3,12, Given the stationary Markov process 
X= (xp 0, of, Pz 4), let a= {a,} be : multiplicative functional 
of X, where a,$,0,==0,,,for any s, t>0. w€Q,,, (a strengthened 
form of requirement 3.22.8). We a 


E(jo)= ae +4 (0)=0} (wEQ)), 


(o E25) (3.103) 
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(if a(o)>0 for all0<t<C(w), we put &(w)=2(w)), The 
function §(w) has the following properties: 


5.23.8, OCE(w)<l(w) (wEQ). 
3.23.B. For every t>0 [:>2}€&,. 


3.23.C, For any s>0 
{ >syS {6 5 =F —s}. 

If ¢(w) is any function subject to conditions 3,23,A-3.23.C, 
the formula 1,=% 5; yields a multiplicative functional of 
X, and the subprocess x = (tp F A, P,,, §,) corresponding to 
this functional is defined by the formlae 

Fw) =F) — (wEQ) 
X;,(w) = x, (w) WEL, 0S t<é(w)) 

oh, = oft, | >t] (3.104) 
of? = of, P,=P, 

GA= (8,4, >t} (AEN) 


(the space of elementary events for X is the same as for X ). 


Proof, The function § defined by (3.103), clearly satis- 
fies inequality 3.23.4. The equation 


{E> t} = {ay > 0}. (3,105) 
yields 3.23,.B, Furthermore, by 3,22,f and (3.105), 


{E> s, 6,5 >t} = [a, > 0, 9,0, > 0} = 
= (28,0 > 0} = (244, >0) = >s-+e}, 
whence we easily obtain 3.23.C. 


Now let = be any function subject to conditions 3,23,A- 
3.23.0, Clearly, %=%:5, now satisfies conditions 3.22.a, 
3,22,Y and 3,22,8, Moreover 

0,00, = Xe> sles i= 4i>s,0>t— 
= kiss, t-s>t— Leo spt Mert? 


so that 3.22.3 is also satisfied, 


Finally, let & satisfy conditions 3,23.A4-3.23.C. The sys- 
tem (x, © of, B,,%,) given by formulae (3. 104) will obviously 
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possess properties 2,8,A-2,8.E and 2,8.G-2,8.H, The series 
of equalities 


B,(,A| of,) =P, (8,4. aa a ee 
=t:5 Ps, (A= Py. (A) (a.c. 2,, P,) 


shows that it also possesses property 2,8.F, The system 
thus defines a stationary Markov process, It may readily 
be seen that X and X are connected by relationships 3.21,A- 
3,21.E (where 7(w)~w). Hence X is the subprocess of X. 
corresponding to the functional {a}. 


3.24. Let X= (x; 6, of; P,, §)be a stationary Markov process 
in the measurable space (E, #), such that &,© o#,, and let 
TCE. In accordance with (3.49) we put 


(TD) = inf {t: x,€ TP}. 


We shall call the set [ admissible for the process xX if 
&(1) satisfies condition 3.23.B, Conditions 3.23.4 and 

3.23.0 are easily seen to be also fulfilled in this case, 
Formulae (3,104) thus yield a subprocess of , We shall 


refer to it as the subprocess corresponding to the admissi- 
ple set I*), 


The transition function of this subprocess is given by 
Bit, x, G)=P,(x,€G, (1) >t}. (3.106) 


We now consider the system (x; & of, B,, 6, defined by 
formula G. 104), the measure P,, being considered for x€T 
only. This system evidently yields a stationary Markov pro- 
cess in the measurable space (I, @{[T))s. We shall describe 
it as the part of process X in the admissible set I'**), 


Theorem 3,13, let £,€@ be a sequence of sets in the 
measurable space (E, #) such that £,+8&. Suppose, further, 
that for every a stationary Markov process X™ we = (x, rn) 
A”, p™, is given in the measurable space (E,, #[E,]), xo) 


*)It must be emphasised that X and I uniquely define the 


subprocess X of xX corresponding to the admissible set I', the 
space of elementary events for ¥ being the same as for x . 
**)As distinct from the general case discussed in sec.3.9- 
3,10, a part of a stationary process X= (x4, %, Mt, Px 02) 

can be generated in every admissible subset, 
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being the part of X@+) in the set £,, Then a stationary 

Markov process X =(x, {, of;,P,, 9,) exists in the space (E, ¥) 

such that C(w)= lim 6™%(w) and X@ is the part of X in the 
n>x0 


set E,. 
Proof, Since X™ is the part of X™@+) in the set £E,, we 
have in accordance with (3,104), 
0 (w) <0" 79 (w), 
ePe@=aP (0) for 0<t <0), 
oli? = oft *? 1 (w) > £1, 
(off) —= (off)"*9 and p® —. p+) for x€Ey 
OA = {HP PA, (> ah. 


Therefore the limit exists: 
Sw) = lim Cw) 
n>c 


and a unique meaning, independent of the choice of n, attaches 
to the expressions 


H()=x(v), if O¢t<e™@ | 
oft, = ott? (6 (0) >] (3.207) 
of = (of); p, =P”, if x€F, 
4={04,¢>¢], ap AEN. 


The system (x; [ of, P,, 6,) is easily seen to satisfy condi- 
tions 2,8,4-2,8,H, so that it defines a stationary Markov 
process X, For this process, the instant ¢(E,) of departure 
of the trajectory from set £, coincides with ¢™, and the 
part of X in set £&, coincides with X™ by virtue of 
(3:107)*). 


All the definitions and propositions relating to subproces- 
ses and parts of stationary processes corresponding to admis- 
sible sets are easily carried over to subprocesses and parts 
corresponding to admissible s:stems of sets, 


*)It may be remarked that ¢ coincides with the instant of 


first departure of the trajectory of X from the sequence {£,}. 


CHAPTER 4 


THE CONSTRUCTION OF MARKOV PROCESSES 
WITH GIVEN TRANSITION FUNCTIONS 


1, Definition of Transition Function, Examples, 


4.1. We take any measurable space (E, #), The function 
Po, x4 Oc<sct, x€E,TE#)is said to be a transition 
function if the following conditions are satisfied: 


4.1.4. P(s, x: 4,1) is a measure, (As a function of the set 
£.) 


4.1.3, P(s x: ¢, I) is a #-measurable function of x, 


4.1.0, P(s, x; t, NH<il. 
4,1.D, Pls. x5 5s, EX x)= 0. 


4.1.B. Pls, x; 4, N= f Pe x 4 dy)PQ vy uw DOSsctcy) 
B 


We shall describe the transition function P(s, x; 4, T) as 
normal if P(s, x; s, E)==1for all s>0, x€E, and as complete 
if P(s, x; t, =1for t>s>0, «€E- 


There is a corresponding transition function for every 
Markov process: 


P(s, x; t, T)=Pe, 2 {*,€T) 


(properties 4.1,.4-4.1.D follow from 2.1.C, 2.1.D, 2.1.5; 
property 4.1.8 follows from(2,12)). If the Markov process 
is non-cut-off, the corresponding transition function is 
complete. 


The aim of the present chapter is to investigate under 


what conditions does the transition function P(s, x; ¢, T) 
correspond to some Markov process. (The connexion between 


96 


MARKOV PROCESSES WITH GIVEN TRANSITION FUNCTIONS 97 


the various Markov processes having the same transition func- 
tions was discussed in article 3 of Chapter 2.) 


4.2. We shall consider some examples of transition func- 
tions, 


4.2.1, let & be a numerical straight line, # the class 
of all Borel subsets of F, and v an arbitrary constant, The 
expression 


P(s, x5 4 yy, [x+ 0(¢—s)] 


defines a complete transition function of (£.¢%). We shall 
describe this function as corresponding to a determinate 


motion with velocity v 


4.2.2, Let E be the set of all natural numbers and # 
the class of all subsets of this set, and let p(s. Oi, j=1, 
2,....m,...; 0 s<r)be a system of fanetions satisfying the 
following conditions: 


4.2.2.4. Pils D> 


4.2.2.3, Zev 0 <i. 

4.2.2.0. piyz(s, s)=Ofor ij, 

4.2.2.D, Dry (s Pat d=pPals. 2) 6<t<a). 

The expression 

P(s, x; t, T)= Di Pow (s: ) (@E€Es<hTe€x#) (4.1) 

now defines a transition function, and it may easily be seen 
that all the transition functions in the space F={1, 2,..., 
n,...}can be got by this method, The necessary and sufficient 


condition for the transition function to be normal is that 
Pu(s.s)==1 . The condition for completeness is given by 


Dpiy(s. = 1. 
jai 


Everything that has been said is applicable also to the 
case when F consists of a finite number of points (1, 2,. 


n}, 


4.2.3. Let E be the n-dimensional Euclidean space ” 
and # the o~algebra generated by all open sets of this space, 
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We write for each x€E,TEF, 


a (£— <7 (y— x)? 
ad a) [ exp [— Sta Y (4.2) 
P(s, x; 4, T)={ fe Fs 
wr) for 0<s=t, 


where (y— x)? denotes the scalar square of the vector y—x 

and integration is carried outwith respect to the usual Lebes- 
gue measure on Rk” It may easily be verified that the function 
P(s, x; t, P)defined by this formila satisfies all the conditions 
4,1,A-4.1.£ and is consequently a transition function, This 
type of transition function is complete and is known as a 
Wiener transition function. 


4.2.4, Let E=(0, co) and @ be the c-algebra in the space 
E generated by all intervals, We shall consider a function 
closely connected with the one-dimensional Wiener function, 


Pi, 54, D= 
oe ee (y — x)? (y+)? 
V 2x raat exp | — 2¢ 7 |—exe[— —e- 5] } ay (4.3) 
bak for 0<s<i, 
Xp (*) for O0<cs=—t. 


This function clearly satisfies conditions 4,1,.A-4.1.D.‘'A 
certain amount of calculation shows that it satisfies 4,1.E, 
The transition function (4.3) is normal but not complete, 


4.2.5. We keep the same values for E and # as in sec, 
4.2.4 and put 


P(s, x; t; Hn= 
eres. eee (y— x)? (y+x)? 
VirG—5 i { exp [—3 2¢— Seay ter [-veal jay 
= (4.4) 
for0<s<t, 
Xr (*) for 0<s=t, 


It is easy to verify that P(s, x; t, T)is a complete transition 
function, 
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4.2.6, Let (E, #) be any measurable space and I(x, Ty (x€E, 
T€ #) a function satisfying the conditions: 

a) U(x,Pis a probability measure on # for any x€F3 

b) W(x, Mis a @-measurable function in E for any TE. 

We define functions IL, (x, Tyas follows: 

Ty (x, Nx), 
TQ N= fA, dy) MrT) (n> 0). 
z 
It may readily be seen thats 
1) 0<U,(x, M<1 for any n>0, x€E, TEs; 


2) Unsm(% T)= f One. dy) Tn (y, I). 
E 
We put 


7 os ~ an (tf — s)% ‘ 
PG REDS Dy Sag 8 tO (eB) (4.5) 


n=0 


where a is a positive constant, In view of 1), the series 
on the right-hand side of (4.5) is convergent, whilst in 
view of 2) »P(s, x; t, P)satisfies condition 4.1.—, It is easily 
seen to satisfy conditions 4,1,4-4.1.D also. We thus have a 
transition function which is clearly complete, It will be 
described as a Poisson transition function, 





2. The Construction of a Markov Process with Given 
Transition Function 





4.3. Theorem 4,1. Let (£, @. #) be a s-compact topological 
measurable space satisfying condition 1.9.A. Every complete 
transition function P(s, x; 1,T) O<s<t, x€E, TE), will 
now correspond to some non-cut-off Markov process in the 
phase space (£, @, #). 


Proof, We shall construct the non-cut-off Markov process 
X in its canonical form, i.e, we take E'(/=[0, co)) as the 
space of elementary events, put ~:(9)=9() ¢€/, 9€EF) and 
write of* for the c-algebra in the space E! generated by 
sets {9:9(u)ET} (u>s, TE GF) and Ni for the s-algebra gener- 
ated by sets {¢:9()ET} (s<u<t,TE #) (see sec, 2.13). We 
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still have to define the probability measures P,, in the s- 
algebras w*°; this is done by using theorem 1,2, 


We first of all use (2,15) to construct in accordance with 
the transition function P(s,*;t,T), the functions 


P(s, x; ty, Ty, sss tas Tr) 
(EE, SStSh<S... Sty Ty ees Pye #). 


We put, for any 4,..., 4,€[s, co), 


O1,.,4, 7p oo P= PS. 4X; fi Th... ti Th), 
where 4i,<4,<... Chi, and (i... i,) is some permutation of 
the numbers 1, 2,...,2, The functions %1,...4, are easily 


seen to satisfy all the conditions 1,12,A-1,12,.C, We use 
theorem 1,2 by putting T=[0, 00), F=(s,00). The theorem 
states that a probability measure P satisfying condition 
(1.28) exists on the s-algebra Wy=of’, The measure P 
depends on parameters s and x, and we write it as P,,. We 
have for any x€EF, 0<s<t<...<ty, Ty---, EF? 


Peo {%2, ET, ery x4, € P,) =P(s, x; t,T,, sees fas r,,)- 


The system (x, of, P,,) obviously satisfies conditions 2,1.A- 
2.1.E. Furthermore, the functions P(s, «; t, Ty, ..., t,, T,) 
satisfy expression lo, 14), and by lemma 2,1, condition 2.17 
is fulfilled, The system (x, oN%, P.,.) thus defines a non-cut- 
off Markov process, ‘The transition function of this process 
obviously coincides with Ps, x; t,T). 


4.4, Theorem 4,2, Let (E. @€. #) be a c-compact topological 
measurable space subject to condition 1.9.4, Every normal 
transition function P(s, x; ¢, fT) will now correspond to some 
Markov process, We can assert, furthermore, that the function 
corresponds to some part of a non-cut-off Markov process, 


Proof, We write E for the set obtained from E by the 
addition of a single point e, Let AcE. We put AC@ if 
ANEEC, and A€ @ if ANNEC #, It is easy to see that 
(E. @. @) is a c-compact topological measurable Space satis- 
fying condition 1 296 A, We define the function P(s, x;#,T) in 
the space €, @, #) as follows: 

P(s, x; t, H= 
P(s, x; 4, TE) +x, @[1—P(s, x; t, EB], (4.6) 
={ if *#a, 
Xp (2). if «=a. 
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The function — is easily seen to satisfy conditions 4,1,A- 
4,1.B, so that it is a transition function, This transition 
function is complete, and by theorem 4,1, a non-cut-off 
Markov process X exists in the phase space E @, #) corres- 
ponding to P. Using lemma 2.4, an equivalent process X of 
the non-cut-off process X can be constructed, for which the 
set E is inaccessible from a, Since the function P(s, x; t, T) 
is normal, F satisfies condition 3.8.B,. In accordance with 
theorem 3,4, we can generate a part of process ¥ on the set 
E, Taking into account (3.55) and (4.6), the transition 
function of the 'part' is seen to coincide with P(s,x:4T). 


3, Stationary Transition Functions and the Corresponding 
Stationary Markov Processes 


4.5. The transition function P(s, x; #,T)is said to be station- 
ary if it depends only on the difference t—s, We shall 
write in this case P(t—s, x, IT) instead of P(s, x; #, TP). 


The functions constructed in 4.4.1, 4.2.3-4.2.6 may be 
mentioned as examples of stationary transition functions, 
The functions of 4.2.2 are stationary when, and only when, 
the functions p;;(s, f) depend only on the difference t—s., 
We shall write o,;((—s)in this case instead of pis(s.f). 

If we take into account conditions 4,1,4-4,1.E, we can say 
that the function P(k,x,T) (h>0, «EF, TE #) is a stationary 
transition function when, and only when: 


can P(r, x,T) is a measure, (as a function of the set 
Tr e 


4.5.B. P(h,x,T) is a #-measurable function of x, 
4.5.0, 0S P(A xT) <1, 


4.5.D, P(0, x, EN x)=0. 


4.5.Be P(t thy x, T= f P(t, x, dy) Pll» ¥ T) (hy hy, D0), 
Ez 


The condition for a stationary transition function to be 
normal is 


P(0,*%,E)=1 (x€2). (4.7) 
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The condition for completeness is 
P(t, x, E)=1 (¢>0, x€B). (4.8) 


In accordance with theorem 2,10, the class of equivalent 
Markov processes contains stationary processes when and only 
when the transition function corresponding to this class is 
stationary. 


4.6. We shall indicate a method by which any transition 
function may be associated with a stationary transition func- 
tion in a rather more complicated phase space, In view of 
the connexion between transition functions and Markov pro- 
cesses, this method enables us to reduce a discussion of non- 
stationary to a discussion of stationary processes, 


Given the transfer function P(s,x,;t,T)(Q0<s<t, «€T TE# 
let (EZ, g)e a measurable space, where F=/XE, 9—=@& Xx g 
(J = [0, co) whilst #2, denotes the ¢-algebra in space /) genera- 
ted_by all intervals), and let the function P(A, y, B) (4>0, 
yCE,BE B) be defined by 


P(t, y, BYP. x th, 1), (4.9) 


where y=(t, x) and the set I is defined by the conditiom z¢€T? 
if ¢+4, 2€B, (It follows from lemma 1.4 that TC€#, so 
that (4.9) has a meaning for any h>0O and any BE#.) It 
follows from 4,1,A-4.1.E that the function Ph, y, B) satisfies 
4.5.4, 4.5.0, 4.5.D and 4.5.8, Im order for 4,5.B also to be 
satisfied, the additional requirement is needed that P(s, x; t, T) 
as a function of s, x and ¢ should be measurable, 


CHAPTER 5 


STRICTLY MARKOV_ PROCESSES 


1, Random Variables Independent of the Future 
and s-Past, Lemmas on Measurability 


5.1 Among all Markov processes we shall distinguish an 
important class for which the principle of the independence 
of the future from the past for a known present is valid in 
a strengthened form, More precisely, in the statements of 
the condition for a process to be Markov it is a matter of a 
fixed instant ¢ (one not depending on chance); for the pro- 
cess to be considered in the present chapter, a similar con- 
dition is also satisfied for a definite type of random vari- 
able (random variables independent of the future and s-past), 


Let X= (xp, 6, oft. Ps,2) be any Markov process, We shall 
call the non-negative function <(w) a random variable indepen~ 
dent of the future and s-past if: 


SLA. s<Kt(w)< maxis, S(w)] (WE 2). 
5.1L.B. fortWwy<ct<l(w)jEoh (sc. 
(Condition 5,1.B can be visualized as meaning that the 


answer to the question, which is the greater, « or f, depends 
only on the phenomena observed in the time intervals [s, mia 


@O].). 


By 5.1.4, {(t>#} U(t+<t<S}=(>12] € of? , so that 5.1.3 
is equivalent to the following condition: 


5.1.B°, fw:r(@)>t}€ of (s<zb). 

We put 2 = {w:t(w) <C(w)}. 

The subsets AC@., such that for any t>s,{Atct<t}e 
oA; , way easily be seen to form a ‘c-algebra in the space Q.. 


We shall denote this c-algebra by o#2. It is natural to 
regard of: as the class of all events observed during the 
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time interval [s, t]. 


If the function « satisfies 5.1.A and takes only a finite 
or denumerable set of different values (4, 4%. ...€[s, co) in 
Q,, it is a random variable independent of the future and 
the s-past when and only when {(t=%,<C}€oM#;, for k=1, 2,... 
With this, A€o#? when and only when {4, c= te < CE of, for 
all k. 


4n example of a random variable not depending on the future 
and s-past is given by the function 


| t,if S()>4, 
t()=)C@), if s<l@<s 
s,if C@<s. 


For this function, @.—@, . Another example is the function 
t(w) = max [0(w), s] lor which &.—9), Further examples are 
provided by the systems of random variables § satisfying 
conditions 2.7.4-3.7.C, on the assumption that Hicos#i*) ; 
and in particular, by the instants of first departure after 
s of the trajectory from the admissible set I (see sec.3.8) 
or from the admissible system of sets # (sec. 3.11). 


The function £(w) defined in ©, is measurable with respect 
to of*, when and only when for any ¢>=s the function induced 
from § in the set (t<¢<} is measurable with respect to 
oft, The measurability of the function § with respect to 
the co-algebra of#% may be pictured as meaning that § depends 
only on phenomena which are observed during the time [s.+] . 


It may be remarked that the function t(w) satisfying condi- 
tion 5,1.4 is a random variable that does not depend on the 
future or s-past when and only when its restriction to the 
set 2 is measurable with respect to oft. 


Lemma 5,1. Lett be a random magnitude independent of the 
future and s-past for the Markov process X, and let Q’€of%. 
Moreover, let the function 7(w)(w€Q2.)be off-measurable and 
satisfy the inequality y>+. Then the function 
{ n() for ©€ 2, = (2’, n<G}, 
bd = = 

WO=| maxis, Co for 68. 
*)This assumption does not seriously affect the generality 
since by sec.2,2 we can always extend the c-algebra o{ to 
om? = #¢ without thereby transgressing requirements 2,1,A-2.1.E, 
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represents a random variable independent of the future and 
s-past, and AQ,»C Ms for any AC of’, 


Proof, Since 17>, we have for any t>s, 
(n<te[r<t} 


and Im*® <E<O) = (Ltt (nt tt <j E Mt. 


The first assertion of the lemma is proved, Furthermore, if 
A€oM; , we have 


{AQ MCE SLA t<tE< EM <t <0] Cot, 
which proves the lemma, 


5.2. ,We put for brevity f=[s, ¢], f=[s, oo), /,= [0. 4] and write 
# # and #, for the s-algebras generated by all intervals 
in f, P and J, respectively. 


The function x(u, o)=+,(w) induces a mapping of the measur- 
able space (Ii X 2. BX of!) into the measurable space (FE, ¥). 
The Markov process is said to be measurable if this mapping 
is measurable for any 0<¢s<t. 


Lemma 5,2. Let X=(x, 6, M%,P...) be a measurable Markov 
process and t(w) a random variable independent of the future 
and s-past, Then the function 8(w)=<«[t(v), ] defines a measur- 
able mapping of (&,, o4%) into (A #). 


Proof, We put C,={t<t<l}. We have to show that, for 
any t>s , the mapping of (C, .o#{) into (E, g) induced by 8 is 
measurable, Since « defines a measurable mapping of (C,, of) 
into (7.9%) , the expression a,(w)= {t(w), 0} defines a méasur- 
able mapping of (C;, off) into (7x2; #1 X off) (lemma 1.3). On 
the other hand, in view of the measurability of the process, 
the mapping a of (Jix 2: #1 X of?) into (E, #) induced by the 
function x(¢,) is measurable, Consequently the mapping %2, 
of (C,, oM?) in (E. #) is measurable, But obviously, a2, coin- 
cides with the mapping induced by fin C,. 


Lemma 5,3. Let X=(x, (. off, P..2) be a measurable Markov 
process and let the function 


P(s. x; t£.T)=P,. {x,€T} (s€f,, x€E) 


be measurable for any >0,°¢€ @ with respect to @:xX#%, 
Then 
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a) whatever ther>0 and A€", the function 
Ps, » (A) (s€l,, x€ £) 
is #,X # -measurable; 


bd) for any 2>1 and any [,...,T,€@, the function 


Pcs, x3 ty Ty, .--> tas Dy) = Pow (%1,E Ti. Hep ETH} 
(8 < ty o.oo ty EE) 
is @°X # X[#]"-measurable. 


Proof, Assertion a) can be proved simply by repeating the 
proof of lemma 2,2 with a few trivial changes, 


We turn to the proof of assertion b). We take the function 


we: o=| P(s, x; t, E)=P, g {0 >t}, if SKt, 
0, if s>t. 

With s and x fixed, ¢(s, x, t)is a function of ¢ continuous 
from the right, With ¢ fixed, it is a ¢°xX #-measurable 
function of s,x« , It follows from this, by lemma 1,10, that 
(s, x,t) , and consequently also P(s,x; t,E) is aSxK @x g- 
measurable function of s, x and ¢t. We complete the defini- 
tion of function yr[x,(o)] by putting Zr[x;()]=0 with wé®,. 
Now clearly, 

P(s, x; ty Ty. ty Th) = 


= fxr ier, ©] --- xen l%n (IPs, 2(da). (5.1) 


We select an arbitrary r>0. It follows from the measur- 
ability of the process that, for any k=1,2,...,, the func- 
tion Xr, El. 0E2) is #"X of’-measurable, The 
function 


Xr, 2, @)] --- Xr, [*:, (#)] (t, wen te El, w€ Q) 


is therefore measurable with respect to [¢']” o#". In view 
of a), the function P,.,(4) is #,X #-measurable for any 

AE . Therefore by lemma 1.7, the function defined by 
(5.1) in the set s€/, x€E,t,....t,€" is#X#XIBT- 
measurable, 


We now observe that, if C¢€ g? and A is the set of systems 
(s, ¥. ty .-++4,) for which 
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{s<tp weal gs P(s, x,t Ty oe ty P,EC}, 


we have 
A= [Ute 1,XEX url] U{A.f==s}U ... U[A t,=s}, (5.2) 


where r runs through all non-negative rational numbers. 


We have by what has been proved: 


{A,X EXUPIE SX BXIPT SSX wx gy". (5.3) 
We now notice that, by 2.1.5, 
P(s, x; s,T)=7, (x) P(s, x; s, B). (5.4) 
Hence with n=1, 


{A, ty = 8} = (ts, 7, (x) Pls, x; t. EEC] E BX BX H, (5.5) 


and assertion b) follows from (5.2), (5.3) and (5.5). With 
n>1, we have by (2.14) and (5.4) for k=1, 2,....23 


{A, tyes) = (tp 55 S Sty ees Feats teats ooo Eni 


Lr, (%) P(S. ¥5 tps E) P (8, X3 ty Ty eee tee Teens +> 
testing iailo le Ch 


If we assume that assertion b) has already been proved for 
the case n—1, it follows from this that 


{4,4 =s}= B°X @X [ By” (5.6) 


On combining (5.2), (5.3) and (5.6), we conclude that 
assertion b) is likewise satisfied for the case a. 


Remark, The function P(s, x; ¢,T) is always measurable 
with respect to x, and the arguments used in proving lemma 
5.2 show that, for any measurable Markov process, the func- 
tion P(s, x; t,T) is measurable over the aggregate of x and ft, 
For the stationary process X=(x,, C, off, Ps,2) we have P(s; x; #, 
l)=P(0,x,t—s,T) by theorem 2,9, Hence for any stationary 
measurable Markov process P(s, x; ¢,T) is a measurable function 
of s,x,t . 
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2. Definition of Strictly Markov Process 


5.3. Definition. The Markov process X= (x; 0. off, Ps,2) in 
the phase space (F, #) is said to be strictly Markov if it is 
measurable and satisfies the following conditions: 


5.3.4, With any ¢>0, TC 9, 
P(s, x; £. T)=Ps 2 {x,€T} 


is a #,X @-measurable function of s and x (#; is the o- 
algebra of the subsets of space /,—[0, ¢]. generated by all 
intervals contained in /,). 


5.3.B. If + is a random variable not dependent on the 
future and s-past, we have for any off:-measurable function 
nw) >t(v) and any +€F,TEC#, 


Pox (x,€T]oM}) = P(x, x57,T) (2.0.2, Psz). (5.7) 
Condition 5.3.B can take the following form: 


5.3.B°, Whatever the x€F, s>0, TE , the random varia- 
ble « independent of the future and s-past, and the offi- 
measurable function 7(w)>t(w), the following equation holds 
for any A€oM: 


Pe o(A, XEN) = f Ps, x5 1 T) Pe, 2 (du). (5.8) 
A 


For, by definition of conditional probability, requirement 
5.3.B is equivalent to the combination of 5,3.B” with the 
requirement that P(t, x,; 7, be anof#:-measurable function of o, 
It follows from condition 5.3.4 by lemma 5.3.b, that the 
function P(s, x; 2,T)is #°X #X *-measurable, The of{- 
measurability of the function P(x, x; y,T) therefore follows 
from lemmas 5,2 and 1,3. 


5.4. Later on we shall construct examples of Markov pro- 
cesses which are not strictly Markov (see sec. 6.18). Con- 
ditions 5,3.B-5.3.B° are not fulfilled for these processes, 
We now show, however, that a weakened from of the conditions 
is satisfied for any Markov process. 


lemma 5,4, No matter what the Markov process X, conditions 
5.3.B and 5,.3.B’ are satisfied if the random variables + and 
71 take only a finite or denumerable set of values in @.. 
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Proof, Let t take the values ¢,,4,..., and 7 the values 
Uy, Us, .. Let AE off: TC # e 


Clearly, 
An= {A t= th, 1= a} = 
=(4 t= 4} (nap th} CoMi, ( R=12 2) 
and by 2,1,.F°, we have for t;:<u, 
Ps, (Ain. ©, ET) = Po, 2 (Ap, Xu, ET) = 


re f Pl X43 Uk P) Ps, 2 (dw) = LPG, x Ts T) Ps, 2 (dw). 
Aun Aix 
On summing these relationships over all pairs i, ek for 
which <u, we get equation (5.8). 


We now notice that the function $(w)=P(z, x; 7,% induces 
in the set A, the function P(é;, x; 4 T), which is clearly 
oM;, -measurable, Therefore, whatever the number a, we have 
for 4%<f8 


{A,. $(o) <a, t<t<j= 
={P(t. x13 ae T)<a tcc} nau, £< SE off. 


The sum of all these sets is equal to [$(e)<atct<t}. 
Hence this latter set also belongs to of}, The function v(w) 
is thus o#2-measurable, and (5.7) follows from (5.8). 


6.5. Condition 5.3.B or 5.3.B’ for a process to be strictly 
Markov is to be regarded as a variant of the principle of 
independene of the "future" (x,) from the "past" (of%) for a 
known "present" (x,), We now show that this principle remains 
valid for a wider conception of "future" (any event, defined 
by the values *;,---» *,, where 7 .... 9, are random variables 
depending only on the phenomena observed during the time 
[s.t]) . A precise formation is given in theorem 5,1, the 
proof of which follows the same lines as the proof of theorem 
2.1 except that certain extra technical difficulties have to 
be overcome, 


We shall first prove a lemma, 


Lemma 5,5. let X=(x, €. off, Ps,c) be a strictly Markov 
process, t a random variable independent of the future and 
S-past, 71> an o#{-measurable function, and (w€2., x€£) 
®(w, x)an of: X #-measurable function such that My, 2P(w, x,) 
exists, Then 
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Mg, 2 (® (wv. x} o43] = Yo % Xe 1) 


(a.c. 2 Ps,2): (5.9) 
where 
YM “4 Y. V)= f Pleo, xX, (w)] Pu, y (dw). (5.10) 


Proof. We suppose first that 
Dw, =Z_ (w)Zp(x) (CE oA, TE B). 


Then by 1.6.F, 
Ms, 2 {© (w, x,)] M2} = zo (w) Ms, 2 (xp (*,)] oM2} = 


=Zo(0)Ps al, ET] of}  @.c.2., Psa) ny 
and on taking (5.7) into account, 
Mg, 2{O(w, x,)f o%:} =x (w) P(t. x5 a T) (5,12) 
(a,c. &., Ps, 2). 
On the other hand, 
$(W HY Y= Xo (Hy) PCs, ys wT). (5.13) 


Equation (5.9) follows from (5.12) and (5.13). 


Let -¥ denote the class of all functions ®(w, x)(@E2,, xEE) 
such that Ms,2?(v, x,) exists, The class & of all functions 
®(w, x) for which relationships (5.9) and (5.10) are satisfied 
is evidently an #-system. The sets CI form a x-system C, 
By lemma 1.2, the .%-system 53€ contains all functions 6 of 7 
measurable with respect to the c-algebra.c(@)=o# x g. The 
lemma is proved, 


5.6. Theorem 5,1. Let X=(xz, 6, off, Ps) be a strictly 
Markov process, + a random variable independent of the future 
and s-past andt<7.---. % a system of o#f%-measurable func- 
tions. Then for any I, ..., TC #5 


Ps, x (x4, ET tee x, ET] os} = 


(5.14) 
=P, Xe Ms Ty wees Mn T,) (a,c,&., Paya)» 
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where P(s, x; tT, ..., t, T,) is given by formla (2.13). 
Proof, With n=1, (5.14) reduces to condition 5.3.B. 
We assume that (5.14) is satisfied for n—1, and prove it 

for n, We put 
4 (©) = min (M1 (0), --., Mm (w)), 
Ar= {1 > 06 Ma > De m=}. 


We notice that A,€o#2, tC n< me ---» taand 7 are measur- 
able with respect to o#;. We fix a value of R(1<k <n), put 


no) for ©€2,,= [Ay 1< 4}, 


* = _> 

a keh max(s, %)for w€2, 
and denote the random variables %---, %, considered only in 
the set 2, by 1 .... 7%. It follows from lemma 5,1 that 


a is independent of the future and s-past and 7[(w),..., 
nf, (v) (o € &,,) are of$,-measurable, Apart from this, the in- 
equalities yi>7° are satisfied, so that by our inductive 
assumptions 


Ps, (Ci Mis) = Ha (a es ee thee Tyg WE 


oli 
SA. y3 Dy eee Naas Mette eee Mn) (Bele Qye, P, x) (5 5) 
where 
Ce {¥t EN, seve Me ET yay | 
1 "k=1 
x « Thane Xe EF, 
ak k+1 oc nf (5.16) 
Aly (@, Ys Uys oes Una) = 
=P (a, y3 Op Ty. es Una Tyets Uses Tests oes Ugeas Pq) 
We put B={x,,€T,,.... ,€0,| . Obviously 
n 
B= 
Ul4 ¥ay € Ue Ci}. (5.17) 


By (5.15) and (5.17), we have for any A€of#", 
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n 
Ps, (AB) = 3 Ps, « {Aw ty, ET Ca] = 
k=1 
n 
=z, f Hy (qe Xqs My oe 
* {Ady 7 eTR} 
Feee Moa Morar ++ Mn) Po, « (do) = (5.18) 


n 
on Mg, rl Pz (o, 4) %4)= 
n 
=2 Mg, 2 {74Ms, ol (®, x,)| oft:]}, 
where 


®; (w, x)= 
= La) An, ) Ae (te 45 Me oes Mente Mest oe 0 Ms (5.19) 


The mapping 2.x F-+/{ defined by the function ,(w,x) can 
be written in the product form fa, where the mappings 
a:2.XE>[PP XE and 8:[h"*> F+f are given by the expres- 
sions 

a(w, x)= 
= (1), M(H) +--+ Mer)» Met (O) +++ Na ()s x}, 
BCD, Ups oes Unis X= AG XK} Vy 6... Vga) 


It follows from lemma 1.3 that a is the measurable mapping 
of (2.x E, of 8) in ([P" XE, 19°)" #). In accordance 
with lemma 5.3, 6 is the measurable mapping of ([/]" x E, 
[f° X #) in (4, g’). The function O,(v, x) is therefore 
Ai X @ measurable, and we are justified in applying lemma 
5.5.- In our case 


P (Wo ws Ys D= f Oylig, x, (0) Py, y (do) = 
2, 


= f 4,00) Xo, #0 ON Hala oa)» XW); tao)» «+ 


se ee Ne-1 Wo)» Neri Mo)» +++ Na (o)] Pu, y (20) = 
ha, (0) f Pla. y5 % dz) Hyln (oo). 25 (oor --- 
T, 


soe Ne 1(Wo)> Ness (o)s -- +> Nn (w)] 
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and by (5,16) and (2.14), 


Y (Wo. 2. Yr 1 (Wy) ) = ha, (0%) P (4s Yi 4 (My). Tre m (). Ty. 
cee Nyt (My) Ty Nhe-+1 (Wo)» Trav +++) Mn (Wo) Ty) = 
=a, (9) P (a ys MHC). Th... Mn (wo), T,). 
Hence 


Y(w, t X,, N= 74, (0) PC x3 My Tyo... My Ty). (5.20) 
On comparing (5.9) and (5,20), we find that 
Ma, « {®x(w, x,)] off] = 
=ha, PC X35 Thy Tr, see An r,) (a.c.2., P,, z). (5.21) 
It follows from (5,18) and (5.21) that 
Ps, (AB) = My oly P( x5 MT te Tye (5-22) 
Since the function P(t. x; m, Ty, .... ”, T,) is o#%-measurable 
(as follows from lemmas 5,2 and 5.3), (5.22) is equivalent 
to (5,14). 
Corollary, Let X be a strictly Markov process, If +, 


Th --+> Ny» ---is the system of w-functions described in theorem 
5.1, and f(*%,,....*,,...)is any #”-measurable function in the 
space E~ , such that My of (Xi. +++» %y,. +++) exists, then 


Me, x {f (*s) seen Kas 4) =F O Keb Hie. Hed Mais 


) 
(2.0.2., Py.) (5.23) 


where 
F(t, Ys Uys voce Ope +) = My, yf (Hoy oo) Loge e+) (5.24) 


Proof, We write -/ for the class of all functions f for 
which M,, ..f(% ---»%:,.---Jexists, and §? for the class of all 
functions f for which equations (5,23)-(5.24) hold. By 
theorem 5,1, -# contains the characteristic functions of 
sets T,X...XI,(#=1, 2, ...; Ty.--.. T,€@). These sets form 
a x-system generating #°, Obviously, 5 is an #-systen, 
and by lemma 1.2, 5 contains all the @@-measurable functions 
of £, 
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5.7. We shall give a further important formlation of the 
principle of independence of the "future" from the "past" for 
a known "present." Some new notation is first introduced, 


We shall write 3° for the z-algebra in the space 2x/’ 
generated by sets 2x J/>(T>s) and 


{(w, 1): ¥5(y(w) EP} 
(T€ #. =( is an arbitrary #*-measurable function in the inter- 
val /° sucn that ¢()>¢ for all t), Evidently, 3° can also 


be described as the minimal s-algebra in space 2x/* with 
respect to which the functions 


f(t, o) =t and O(f, 0) = x, (2 (w)» 
are measurable, 

Lemma 5,6. If T>s and {(w) is an arbitrary W7-measurable 
function, the function 7), 7 ()§(w) is §*-measurable, All sets 
of the form 

(“uEI} X[s, T] (>, TE #) (5.25) 
belong to the s-algebra §*, In conjunction with sets 2 x /} 
(T>s) they generate the s-algebra §*° in the case when X is 
a process continuous from the right in the topological 
measurable space satisfying conditions 1.9,4-1.9.B. 

Proof, In accordance with lemma 1,5, 

EQ) =f (Xe. 26+) Xa o -): 
where /,,..., ty, ...Tand f is a #”-measurable function in 


the space E°. We take any g%,-measurable function 3; (t) 
satisfying the conditions 


GO=t, for s<t<T, 
giQ>t for t>T. 


Obviously, 


Us, qT) @§@) = Ls, 7} Qs (sz, (ft) eer 2 Xe (Oe e+ yy 


so that 7%, 7@)&(w) is J*-measurable, 
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Putting '=y,(«,) (4>T), we conclude that 


{x,ET} X fs, TET. 


Now let X be a process continuous from the right in the 
topological measurable space (F, @, #) subject to conditions 
1.9,4-1.9.B, We write §* for the s-algebra in © x /* genera- 
ted by sets 2x J/;(T>s) and set (5.25). Let ¢(f) be any 9 - 
measurable function satisfying the inequality +(f)>+, and 
let the points ¢{ define a canonical sequence of subdivisions 
{At} of the interval *, We put 


ent, if e@EAM. 
For any TC @ 


[@ ©), @@ET} =U {x e€T] xle Meares. 


The function ~.,()() is therefore 3*-measurable, As noo, 
i()}¢(¢) and in view of the fact that X is continuous from 
the right, ,(9 >, By lemma 1.9, *,()(v)is also 3*- 
measurable, Therefore #*— gf, 


5.8. Theorem 5,2, Let X be a strictly Markov process, 
and < a random variable independent of the future and s-past. 
Let ®(w, t)(wE2, t€/*) be an g§*-measurable function such that 
@(, t)is P,,-summable, Then 


Me, 2{®(w, )JoM} =F x, x) (@.c.2, Pg) (5426) 


where 
F(t, y)=M, ,O(o, f). (5.27) 
Proof, Let 
B= [WS <SEST. Ky MET 6s Xp, ETa}, (5-28) 
where T>s,T,, ....T,€ @ and 9, ..., o, are #*-measurable func- 


tions such that 9,(f)>t. If Ow, t)—=x,, we have 
D(%, = Ho, 7) O dp, wl --+ Xr, [Hen 


1.6.F 
Hence (see ) Ms, 2{® (, 1) fo48} = 


Ey en ee be oe Ue Oe G T,[oM#} (a.c. 2. Ps, 2)s 
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where y,=6¢,(c). By theorem 5,1 it follows from this that 
My, (PB (0, DfoM8} =X, g 7? (X45 MW Ty + Mee Tn): (5.29) 


On the other hand, 
My ®(, 4) = he, 7 Pa yo. Tp -- +) On (4. Ta): (5.30) 


On comparing (5.29) and (5,30), we conclude that relation- 
ship (5.26) is satisfied for ®(.)=x%3 . Sets B of the 
form (5,28) form a x-system @:, The functions ® for which 
(5,26) is satisfied form an ¥-system (. denotes the class 
of all functions © such that ®(w,)is Ps,2-summable), By 
lemma 1.2, this system contains all the functions of 
which are measurable with respect to o(@)=3J*, 


Corollary. let X be a strictly Markov process and let 
the random variable + independent of the future and s-past 
satisfy the inequality «<T. Then for any o’7-measurable 
and P,,.-summable function §&, 


My, (¢1 o#f) =Mz2§ (a0, Py, 2). (5.32) 


This may be seen simply by applying theorem 5.2 to the S*- 
measurable function 4, 7 (#)&() « 


Remark, In an important particular case theorems 5,1 and 
5.2 and their corollaries are applicable to any Markov, and 
not merely to strictly Markov, processes, This is the case 
when the functions +. 7m, ..., 7, take only a finite or denumer- 
able set of values in 2., For, in accordance with lemma 5,4, 
conditions 5,3.B-5.3.B’ are fulfilled for these functions, 
so that all the proofs contained in sec. 5.5-5,.8 remain 
valid, 


5.9. As an application of the general theorems just proved 
we shall mention some properties of the random variables 
5,(1).&, (#F ), t,(G)defined in articles 2-3 of Chapter 3. 


Lemma 5,7. Let xX be a strictly Markov process continuous 
from the right in the topological measurable space (E£, @, #) 
and satisfying conditions 1.9.A-1.9.B. let ¥# be a normal 
system of subsets of (E, @. #) and §,(#) the instant of first 
departure after 't' of the trajectory of the process from 
the system F (see sec, 3,11), Then the functions §&(¥, «) 
G@)>t>s) and 2,7, 0) (w) are J*-measurable, 
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Proof, Let {I,,} be a sequence satisfying conditions 3.12.A- 
3,12,C and equivalent to #. We observe that 


GF wo) = lim i E+ hr) (a) >t>9) 


where A is the set of all non-negative rational numbers and 


+o, if +1 @)ET,, 
1 for the remaining(u, d). 


Sar = { 


The functions f,,,(, ) are easily seen to be §*-measurable, 
This means that the functions (+ 7)/fn,-(w, f), and hence also 
the functions &(¥,w) are S*-measurable, 


Since the mapping of the space (2x/*, $*) into (E, @) defined 
by the function x,(w) is measurable, we can prove that *% (7, «) 
(w) is measurable simply by showing the measurability of the 
mapping f of the space (Qx /*,J*)into itself given by the 
expression 


Bo, N=, &F, w)). 


By lemma 5.6, we can show this simply by verifying that 


82x HET (T>s) 
a {ix,€T XM} CF? (u>T>s). 


The first of these inclusions follows from the §*-measura- 
bility of the function £(%,w) proved above; the second 
inclusion follows from 


B'{ixwET] X fr} = (xu€Ts &(F)<T} = 
={x,EP X fn (F< 7}. 
This proves the lemma, 
Theorem 5,3. Let X be a strictly Markov process contin- 
uous from the right in the topological measurable space 
(E. @, #) and satisfying conditions 1.9,A-1.9.B. We put 
Ta(s. x A, T)= Py a {te(GVEA. x2, ET}, (5.32) 
where GE CN # and ,(G) is the random magnitude defined in 


118 THEORY OF MARKOV PROCESSES 
sec, 3,13, Then we have for any random magnitude 7 not 
depending on the future and the s-past: 


Ps, {t, (GVEA x, € TloM} =Me lm x3 A. T) 
(a.C.2,, Py 2) 


(5.33) 


If VEe@N*#, the closure of U being compact and contained 
in G, we have 


Ps, 2 {ts(G)EA, x2, @) ET] oA} = 
=Te (=U) x, AT) (2.0.2, Ps, 2) (5.34) 


and 
IIe (s, x; A, H= Lie x; dt, dy) IIg ¢, M A, T). (5.35) 


When the process X is continuous, relationships (5.34) and 
(5. 35) are satisfied for any open measurable sets UcG, 


Proof, In view of lemma 5.7, the function 
O(t, ©) = [1G Ol 7p [%y,@ » )] 
is S*-measurable, If we apply theorem ‘6 .2 to this, we arrive 
at relationship (5.33). Equation (5.34) follows from lemma 
3.9 and (5. 33). In the case of a continuous process, we can 
use lemma 3.10 and relationship (3, 54) instead of Lemma, 3.9. 
3. Statio Strictly Markov Processes 
5.10. Let X= (x; %, of}, P,~) be a stationary Markov process 


and let 4, be operators satisfying conditions 2,5,A-2.5.C. 
We take any non-negative function +t() and put for any AEN": 


1.A= Y tia. =) =). (5.36) 


It follows from 2,5,A-2.5.B that operators 6, possess the 
following properties: 


5.10.A, 6.2,=2.,6,(A\ B)=6,AN 6B, 


6. Ua) =Us.4,. 6.14) =M0.4, 


(s runs through an arbitrary set of values), 
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5.10.3, 6, ! x, ET; = {[Xeaa€ ry. 


Let é(w) be an c\"-measurable function, We define the func- 
tion 6: by the expression 


6.5 (w) = 6,5 (w), if tw)=t. (5.37) 
It may readily be seen that properties 2.6.A-2.6,.B8 and 


2.6.G of operators §, still hold when t is replaced by 
In particular, 


° 


$= os. (5.38) 


Lemma 5.8. Let xX be a stationary Markov process and let 
5(w)(wE Q,) be an arbitrary J°-measurable function, Then the 
function W(w, t)=6(wjis measurable, 


Proof, Let # derote the class of ali functions i(u)(wE2,). 
Obviously, the class 5¢ of all functions ¢ for which the 
assertion of the lemma is satisfied is an .Y-system, Further- 
more, the characteristic function /z of any set B={x,€T,,..., 
x, ET} Gp... t, 20; Ty ....T,€#@) belongs to #, These sets 
form a s-system generating the c-algebra A’, The statement 
of the lemma therefore follows from lemma 1.2, 


On combining lemma §©.8 and theorem 5,2 and taking 2.5.C 
into account, we arrive at the theorem: 


Theorem 5,4, Let X=(x, %, off, Ps,,) be a stationary 
strictly Markov process, and < a random variable independent 


of the future and s-past. let & be an oW-measurable function 
such that M, ,0.6 exists, Then 


Mg, 205 )o#)=M,=*) (a,c, 2., Py). (5.39) 
We have for any BES: 
Py, (@,8fo#%)=P,.(B) (a,c. 2., Py, »). (5,40) 


Corollary 1. Let the assumption of thecrem 5.4 be ful- 
filled, Then for any 4€c#2, BE d\"': 


Py (As 0:8)= f Pz.(B)Py, (Wt). (5.41) 
A 





*)We write Mt instead of Mo. and p,(8) instead of Pu, 2 (8). 
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If the function ¢(w) is WN'-measurable, the function 7(w) 
o#i-measurable and 76,: and 9: are P, ,-summable, then 


Ma, (78.8) = Ms, 2 (1Mz..)- (5.42) 
Corollary 2. If vw, 4) (EQ, ¢E(0, 09)) is an arbitrary 
WN X #,-measurable function and if P(v. = (v4), we 


have for any random variable = independent of the future and 
S—pasts 


Me, a {® (, +) | 042} =F G x) (a.c 2., P,, a) 


where 
F(t, y)=M,}(o. 0). 

These expressions are deduced from (5.39) by normal use of 
lemma 1.2, 

Theorem 5,4’. Expressions (5.39), (5.40), (5.41) and (5.42) 
likewise hold in the case when BEAN and ¢ is ,\-measurable, 
(Measure P, is to be continued into ¢# as described in sec. 
2.2). 

The proof of this theorem is very similar to that of theo- 
rem 2,1, We shall therefore omit the details and just give 
the broad outlines. Let A€ of, Bes. We take the measure 
in #: 


eM)=P2(A EN) (PE #) 


and choose 8,, B, from of such that 8, |= BS B, and P, (By) = P, (Ba). 
On applying forma (5.41) to A, B,; we get 


P, (A0,B; = Pz (Bi) Pa (dw) = f P, (B;) » (dy) = P, (B,). 
E 
This gives us 
P, (A9,B) = P,, (B). 
On the other hand, 


P, (B;) = f Px, (B) Px (do), 
A 
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and Spe is proved for sets 4, B., Expressions (5.39), 
(5,40) and (5.42) are deduced from (£.41) with the aid of 
arguments which will by now be familiar, 


5.11. Let & be random variables satisfying ccnditions 
Del A-3.7.C and (3.90). We have 


bio =—t—s (WER). (5.43) 


Moreover, for any aE: 


so that 


(5.44) 


Let f(t, x) (¢(>>0. x€E) be an arbitrary #0.«)X #-measurable 
function, It follows from (5.43)-(5.44) and 2.6.8 that 


Bf Cio: Xin) =f (CE, — Xz.) (5.45) 


We now make the assumptions that process X is strictly 
Markov, that « is a rendom variable independent of tne future 
and s-past, and that the function M, ,f(G—<.«;) is P,.- 
summable, On applying theorem 5.4, (5.45) now gives us 


Ma, be if (G ar xz )f At:} aad Myf Eo: X25) 
(a.c. 2.,P,, .)#). 


(5.46) 


Let G be an admissible set for process X, 


*)In order to apply theorem 5.4, we have to verify that 


function / (%,x,,) (~€Yo)is of’-measurable, In view of 5.7.3, 
{o> tex? for any ¢>50 . The function &(o)(e%) is 
therefore a random variable independent of the future and 
Q-past for the process (+;,£, #1, P.,2). By lemma 5.2, «, is 
measurable with respect to #{. Since #9 and 2” are 
contained in .\, it follows that & and “4 are measurable 
with respect to Wf=\[u,] , whence we may readily deduce the 
wW-measurability of fo, *:), 
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We introduce the following notation: 


eee ye i eG) P (dw) ( > 0), 
*@st 


Ts f (2) = Mge* Of [xgigl= [xb(x. dy) f(y). 
Ez 


Tq(x T= 73(x, TI) =P, {xsa)ET}, 
Hg fo) =I f= MSI, l= [re 410). 


2 
Mg {x)= M,5(@). 


Theorem 5.5. Let U,G be admissible sets for the station- 
ary strictly Markov process X, where US&G. Then the 
fcllowing relationships hold: 

Te f (x) = MeN f (x) (5.47) 


(if e-¥@f[x:q@]is P,-summable), 


na (x, T)= [ xy(x, dy)o(y. 1), 
‘ J 7 F (5.48) 


mg (x) = My (4) Mog [X; yl = 


= my (x) + fm) Ty (x, dy) (5.49) 
E 


(if me(x)<oo). 

Proof, We put in forma (5.46): s=0, f(t, x)=e— f(x), 
&,=§(G), «=5(U). On taking into account the fact that —&= 
E(Gyby virtue of (3.54), we have 

Mz [e* # S01 f Lx, 1) 1] oA9 ny} =H Fx q¢qy] e¢ Lecm,Pa)- 


Since the function (mM is oMz (7) -measurable, it now follows 
that 


M, {e-} £(@) Fixe (foe zm} = e~* (0) We f (xe (my) 
(a,c. 2m; P,) 


On taking the mathematical expectation of both sides, we get 
(5.47). With f()=y, (x), relationship (5.47) becomes (5.48). 


Relationship (5.49) follows from the series of equations: 
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Mf (G)— Mat (U) = f [8 (G)—§ UW) P,, (do) = 
240) 
= [9% (G) Po(do) = My Ma, &(G). 
(uy) 


Remark. Let X be a stationary strictly Markov process 
continuous from the right in the topological measurable 
space (E, @. #)and satisfying condition 1,9.B, and let U. G 
be open measurable sets in (E. @, #), the closure of U being 
compact and contained in G, If we replace §(G) by 1,(G) in 
the definition of functions Iléf(x), «4(x,T), and mg (x), rela- 
tionships (5.47)-(5.49) remain valid, (This follows from 
lemmas 3,7 and 3,9.) 


5.12. We shall now indicate briefly how our definitions and 
results are modified if the stationary process is understood 
as the system (x,, €, of, P,, 9,) subject to conditions 2,8,A- 
2.8.0. 


We shall describe the process X= (x, C, of, P,, 6,) as measur- 
able if the mapping of the space (2; XJ, o4;X #,) into (E, #) 
defined by the function x,(w)is measurable for any ¢, We 
shall describe the non-negative function <(w)(w€Q2) as a ran- 
dom variable independent of the future if +(w) <C(w) and 
{t<t<f}€o4#, for any t. If AGCQ.={r<t} and [A,c<t<¢} 
€oM,for any t, we shall put A€o#,, All the lemmas of 
article 1 are easily carried over to the present case, In 
particular, we can assert (see the remark at the end of .sec. 
5.2) that, for any measurable stationary process, the transi- 
tion function Pi, x, I) is always a measurable functimof {tx}. 


The process X==(x,,C, of; P,,. 9,) will be described as strictly 
Markov if we have for any random variable + independent of 
the future, for any non-negative of#.-measurable function 3(w) 
(~€2) and for any TE#: 

Pa {tas€PJod,} =PC, xT) (a.c.2,P,). (5.50) 
It follows from this that, for any BC: 
P, {9,Bfo#,}=P,,(B) (2.c. &, Pz) (5.51) 


and for any d-measurable function § such that 6— is Pz- 
summable:s 
Mz {6.£Jo#,} = Mz 8 (a,c, 2, P,). (5.52) 
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We remark in conclusion that all the expressions deduced 
in article 3 for stationary strictly Markov processes hold 
for all stationary Markov processes if the random variable 
* takes only a finite or denumerable set of values, 


4, Weakening the Form of the Condition for Processes 
Continuous from the Right to be Strictly Markov 


5.13. Throughout the remainder of Chapter 5 we shall con- 
sider Markov processes continuous from the right in the topo- 
logical measurable space (F, @, #) and satisfying conditions 
1.9.4-1,9,B. We can weaken for these processes the formula~ 
tion of the strictly Markov property 5.3.B by limiting the 
classes of random magnitudes +t and y appearing in the state- 
ment; also, we can conveniently introduce the random varia- 
bles f(x,) instead of the events {x,€I}. We agree to write 
C for the class of all bounded contimious measurable func- 
tions f(x) in the space (FE, @, #). 


Lemma 5.9. The Markov process continuous from the right 
X=(x, 6 off, Py, 2)i8 measurable, 


To prove this, we simply apply lemma 1,10 to the interval 
A==[s, f], the measurable space (Q,, offi), the topological 
measurable space(E, @, #)and the mapping x, (w). 


Theorem 5,6. Let X= (x, 5, of, P,.) be a Markov process 
continuous from the right satisfying condition 5.3.4 and the 
following condition: 


5.13.4. Whatever the x€F, O0¢s<t, fEC, the equation 
holds for any random variable «<t independent of the future 
and s~past: 


Ms, o{f@DIM=M., 0 f(%) @.c.2Py,2) (5.53) 
Then X is a strictly Markov process, 


Proof, The process is measurable by lemma 5.9. let fec, 
let t be any random variable independent of the future and 
s-past and % an of;-Measurable function satisfying the 
inequality 12>, Let the points #2 define a canonical 
sequence of subdivisions {Aj} of the interval [s, oo). We 
put iy 2 ( t(o), = ift@) <4, 

a, if <(w) > #. 
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In view of condition 5.13.A, we have for any ick: 


Ms, 2 {1 (*,n)[ #9} =F, ¥ om i] @.C.Q.n Ps, 2), (5.54) 
where 
F (4, y. 0) = My, yf (%»)- (5.55) 
Let 4€o#2. Then the w-set 


A= {A. t€A?, 1€ Az} 
belongs to ohm , For obviously, Afx€oM; ands? (w)=<(w)for 
wEAn,. Theréfore 
(Ah, P<E<Y = lA, tt <5} CoM 


Since Ay, € ofa, it follows from (5.54) that 
+ 
St(#e) P,, - (do) = fF( ty t)Ps,2(@0). (5,56) 
4h At 
We put Tn (4) = te if 7 (w) Az io Obviously, tn(w) = fh, (w) = tf (w) 


for w€A%,. Relationship (5.56) can therefore be rewritten 
as 


ff) Pe o(de)—= [Fis x w)Ps, ode). 
4k a 


Ak 


On summing over all pairs of values i<k, we get 
fF Gn) Ps, 2 (do) = f P(e 2 19) Ps, 2 (do). (5.57) 
A 4 


We now observe that, since the process is continuous from 
the right whilst f(x) is continuous and bounded, the function 
of (5.55) must be continuous from the right with respect tov. 

On the other hand, 7,}7 as 2-+co so that 

f (Xn) > f %)- 
On passing to the limit in (5.57) as n+oo, we get 
f FG) Po, odo) =f F (Xo 1) Ps,o(da)- (5.58) 
4 A 
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Let denote the class of all bounded #-measurable func- 
tions and let f€ Sif relationships (5.55) and (5.58) are 
satisfied for f. It follows from lemma 1,8 that W297. 
Relationships (5.55) and (5,58) therefore hold for every 
function f€E.%, If we put, in particular, f—yxz,, we get 
equation (5.8). 

5.14, Theorem 5,7. Let X=(x, 6 of*, P,.) be a Markov pro- 
cess continuous from the right satisfying condition 5.3.4 
and the following condition: 


5.14.4. Whatever the x€F, sp 0, kh>0, fEC and the random 
variable < independent of the future and s-past, the equation 
holds: 


Meo (f(%:s))oMe] =F x t+h)  (a.0.2,P,,) (5.59) 
where 
F(a y, 0) = My, yf (Xp)- (5.60) 
Then the process X is strictly Markov, 
Proof, It is sufficient to show that condition 5.13.A 
follows from 5.14.4. let fCC and let + be a random variable 
independent of the future and s-past satisfying the inequality 


7<t, We take the canonical sequence of subdivisions {Aj}. 
of the interval [s, t] defined by the points 


h=stAg—s @=1,2,.....0 m=1,2,...). 
We put 


het —ih., 
Bn(u)==a-+he, if “EAR. 
By (5.59), 


Mo, o{f(*-+np)[oMt] =F (ex. <8) (2.0.2, Py, «)- (5.61) 


Let A€o#;. Then Ap=(4,t€AR} CoM; , and by (5.61): 


SF (0 ») P,, 2 (dw) = fre Xq, TAR) P,, a (dw). (5.62) 


42 an 


But §,(7)=t+AE for o€ Az. On substituting this value in 
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(5.62) and summing over k, we get 
ffl o1Ps,2(du)= fF it. x. BPs, 2(de). (5.63) 
A a 


We now notice that 8,(u)|z¢ as noo, Om passing to the 
limit in (5.63), we get 


f feo Ps 2 (dw) = fF (t. x 2) Ps, 2 (dw). (5.64) 
rs ri 


It follows from property 5.3.4 that for any # F(u, y, this a 
measurable function of{ «, y}so that F(x, M=M, 2. f(x) is an 
oA? -nmeasurable function of w, Relationship (5.64) is there- 
fore equivalent to (5.53). 


Remark, Let Q be an arbitrary family of monotcnic trans- 
formations of the interval [0, w) into itself, satisfying the 
following conditions, 


5.14.0. y()>rfor any 7€Q,4>0. 


5.14.8. The constant K exists, such that, for any 7¢Q, 


s, t,20 ? 
lg@Q—e(s)i<Klé—s|. 


5.14.7. For any 0¢s<z there can be found a ¢eQq such 
that ¢(s)=t. 


Theorem 5,7 remains completely valid, and no substantial 
changes are needed in the proof if we replace condition 
5.14. by the following more general conditions 


5,14.a°, For every €Q: 


Me, x {f leq q Wo#8} =F (ts tn 2 (2) (8.02 Ps, 2) 


(347 have the same meanings as in the statement of condi- 
tion 5.14.a). 


Condition 5.14.a° reduces to 5,14,« when the family Q con- 
sists of the functions ¢(@)=/-+4 (h being any non-negative 
constant). 


It may also be noticed that condition 5.14.a° is equivalent 
to the requirement that condition 5.3.B be satisfied for the 
random variables 1=¢9(*) (%€Q). 
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5. Strictly Markov Subprocesses 


6.15. Is the strictly Markov property preserved when pro- 
cesses are transformed as described in Article 3 of Chapter 2 
and in Chapter 3? It is easily verified that, if a strictly 
Markov process ¥ is subordinate to a measurable Markov pro- 
cess X (see sec, 2,11), X is also strictly Markov, The 
converse does not hold: a non-strictly Markov process will 
be constructed in 5,22 by widening the basic c-algebras of a 
strictly Markov process, We are still less entitled to expect 
the strictly Markov property from arbitrary subprocesses of a 
strictly Markov process, It will be shown, however, that 
given very broad assumptions, the class of equivalent sub- 
processes of a strictly Markov process contains a strictly 
Markov process (this being the canonical subprocess described 


in sec, 3.3). 


The fundamental result of the present article is stated in 
theorem 5.8, We shall first prove some lemmas, 


5.16. Lemma 5.10, If the strictly Markov process X satis- 
fies condition 5.3.4, this condition is satisfied by every 
subprocess of x subject to requirement 3.5.8. 


Proof, Let TEM, O0<s<u; s<t, x€E. We put 


FEH= M, 42x, (x,), if t<u, 
M, .0 Xp (x,).if t>4a. 

Let A be any positive number, We select points r,—0, 

Ty oes PRU such that 
aN Sy eS Hh, ONS, 

We consider the function F(s, x, #) in the set I%} xX EX 

441 
tt *), It is continuous from the right in ¢ (by virtue of 


fr. 


3.5.8) and #% X #-measurable with respect to s and x (by 
lemma 5,3). 1 is consequently (see lemma 1,10) gt x eX 
"S41 


P 
#,{*) -measurable on the product set (whose elements are the 
triples) {s, x, 1}, We conclude from this, on the basis of 

lemma 1,3, that F(s, x, sth) is a Be X #-measurable func- 


. 1 
tion of s and x on the set ih XE@=0, 1,2, ..., R—2)° 


*)We put =[s,2) and write @{ for the c-algebra in the space 


f generated by all the intervals contained in /, 
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This function is independent of s on the set TPA and is 


clearly also measurable (see lemma 2,2), Hence F(s, x, s+h) 
is ag2X g-measurable function throughout the set RXE. 
By lemma 3.1, 


Bis, x; 4,T)=F(s, x, s) == lim F (s. x, Sth) (SER, x€E&). 


Hence P(s, x; u, This a 1X #-measurable function of s and 
x, and the subprocess satisfies condition 5.3,A. 


Let X be a subprocess of the Markov process X, Lett be 
a random variable independent of the future and s-past for 
process X, We shall show that <= min(c, ia) is a random vari- 
able independent of the future and s-past for the subprocess 
X. In fact, by 3.1.0’, 


it <t<ta [rect t< Fe of, 


so that condition 5,.1.B is satisfied, Condition 5.1.4 is 
obviously also satisfied, We observe that AQ~ ¢o#* if AEok. 
For we have for any t>0 : 


{4,2 TEC CHA tt > C > df ofl. 


Lemma 5,11. Let X be the canonical subprocess of the 
Markov process X (see sec, 3,3). Then an arbitrary random 
variable z independent of the future and s-past for the sub- 
process X is equal to min (c, f) , Where t is a random varia- 
ble independent of the future and s-past for process X, Any 
event A€o#! may be written in the form A=Af Q-., where 
AEoh, 

Proof, Conditions 3,3,4-3.3.B define ©,0 x, of? and off® 
for the canonical subprocess. We write C, for the aggregate 


of X4€[0, co) such that (w, \) C2~, and Q, for the set of all 
»€2 for which C, is non-empty, We put 


sup t(w, ’) for w€@., 
t(w) = rec, (5.65) 
max [s, €(w)| for w€ &.. 
We have for every t>s 3 
{T<t< PF} Cot 


so that 
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[F<t<C]=BX@, oo}, (5.66) 


where -BE€of#;. On comparing this equation with the obvious 
inclusion 
(K<t<} XG olelt<t<f}, 


we get 
{¢<ct<QcB. 


On the other hand, it is clear from (5,66) that, if €8', 
we have 7(wy 1) <t <2 (wo, ))for allk>+t. Hence it follows, in 
view of (5.65) and 3.3.B, that <(w)<¢<‘5(u) . Thus 


{r<t <0} = BE oft. (5.67) 
We have from (5.66) and (5.67), for any f>s , 
(Tetctalectcttchalrcrct}. (5.68) 


It follows from this that 


and 
~ 
od 


anh eg 
= min(-, 6). 


It is evident from (5.65) that « satisfies condition 
5.1.4. In view of (5.67), « satisfies condition 5.1.B. 


The function « is thus a rendom variable independent of 
the future and s-past. 


Moreover, if A€o#%, we have for any t>s: 


(A, t<t <7} € of 
so that 


(A. F<t<l}=DX&, oo}, (5.69) 
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where. D€ of? . From (5.68) and (5.69): 
{A,etct tor =(p, tod}. (5.70) 
We put w€A-if (, )€A for all sufficiently large values 


of 4, It is clear from (5,70) that, for »€D,(w.4) € (4, 7<# 
<t} for all }>180 thatwé (A,7<t<%}, Thus 


De[At<ct<f}. 

On the other hand, if »€{4,¢<4<¢}, we have (% WE{A, < 
<t<t, t<d\}={D, ¢<)} for all sufficiently large 4, so that 
#€D, Therefore 

{A,7<t<0} = DE offi. (5.71) 


This shows that A€o#?. It is clear from (5.69) and (5.71) 
that 


{4. 7 <tctl ela cct<?} 
so that 
tpe 


< 
= U {4, tt <t}={a, t<C] = Age. 


Lemma 5,12, If a—({2j} is a mltiplicative functional and 
& a random variable independent of the future and s-past for 
the Markov process X, the function a is of#*-measurable*) . 
If X¥=(x, T, off, P,,.) is the canonical subprocess of X cor- 
responding to the functional a, we have 


Boo (s<Ffo#}=2t @.0.2,P,,). (5.72) 


Proof. let s<t,. The function ai(w) (WE /;,%EQ,) is con- 
tinuous from the right in 4 for any o€@, andis of#i- 
measurable in w for any 4“¢€/;. By lemma 1,10, this function 
is #1 ofj -measurable on the product-set (uz, ). On the 


*)The s-algebra a! is constructed in accordance with the ¢- 


algebra -#® just as Kt is constructed in accordance with -4? 
(see sec, 2,2), It is easily seen that Ac.#* when and only 
when {4,1<1<t}ecés for any 7. 
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other hand, the restriction of the function +(w) to the set 
few) <t<t rw)! defines an of;-measurable mapping of this set 
into the segment /? . The restriction of a ()(w) to the set 
{z(w) C<é<i()lis thus of} -measurable, which proves the first 
assertion of the lemma, 


Further, let A€o#*, We put 8={A,7:<f}. Then in the 
notation of sec, 3.6: 


B =| (¢(e), co] for w€ Ang, 
= 0 for w€ANg@,. 
Therefore 
Op = Oe =O oha (@E2) 
and 


P 2 (B)=M, .24—= M, ahs 


which is equivalent to relationship (5.72). 


Lemma 5,13. If c= {oz} is a multiplicative functional 
satisfying the condition 





5.16.4. Hie oe (sh a5 (a) for all 0<¢s<t<C(w), 


then the function 
f(a, o)=at() WE [s, t], ©€ 2) 
is 3*-measurable, 


Proof. Let the points 7“ define a canonical sequence of 
subdivisions {A%} of the interval [s, #] (see sec, 1.9). We 
put 


Qn(u)= te if uE Ar 
By virtue of condition 5.16.4, 
. : 
lim, 4° (a) =a} (0) =F (, 0). 


It is readily seen t at 
afo  (o) =D Yen gy (0). 
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It follows from this, in view of lemma 5,6, that the function 
as” «) (wo) is S*-measurable, The limit function f(z, ) is there- 
fore also J*-measurable, 


5.17 Theorem 5,8. Let X be a strictly Markov process con- 
tinuous from the right in the topological measurable space 
(E, @, #) and let a= {as}, be a multiplicative functional for 
process X satisfying condition 5.16.4. Then the canonical 
subprocess X of xX corresponding to functional a is a 
strictly Markov process, 


Proof, Since the subprocess X is continuous from the 
right and, in view of lemma 5,10, satisfies condition 5.3.4, 
we only need to show that it also satisfies condition 5.13.4 
(see theorem 5,6). Let fEC, T<t be a random variable inde- 
pendent of the future and s-past for X., and A¢o#%. We 
consider the corresponding random variable + and event A 
constructed in lemma 5,11, It is clear from (5,65) that 
<«<t. We have by lemma 3,1: 

M, of @J=M, XS OD ee, p=, aka *) 4. (5.73) 

By 3.5.B, =o}, and we obtain on taking lemma 5,12 and 
1.6.H into account: 


M,, .Xaf (*) a= M, , (x428M,, 2 (f (x;) af o#s}}. (5.74) 


It follows from lemma 5,13 that the function f(«) at (4 E Is, t}, 
w€2,) is §*-measurable, and by theorem 5.2: 


Me, 2 {f («aif oA} = F(t. x.) (8.0.2, Poe) (575) 
where 
F (4, Y) = Mu, of (x) af = Mu, uf &D. (5.76) 
We conclude from (5.73), (5.74) and (5.75) that 
M,, af &)=M, 2Xa0tF Go %,). (5.77) 
On the other hand, in view of lemma 5,12: 


M, ata (* x) = M, aah (* x) te re a) 
md M,, altar (, x,) P, a [t ae | o#*]} = M,, aka? cc x,).az. : 
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Comparison of (5.76), (5.77) and (5.78) gives us (5.53). 


The reader will easily prove the following variant of 
theorem 5.8 for stationary processes in the sense of sec, 
2.8. 


Theorem 5,8° . Let X=(x,,%, of, Pz, 4,) be a stationary 
strictly Markov process (see sec, 5,12) and let a bea 
multiplicative functional of X for which condition 3,22,8 
is satisfied in the following strengthened form: a,6,a,=:4,,, 
for all s,f>0 and w€®,,,, Then the canonical subprocess 
corresponding to a is strictly Markov, 


6, Criteria for a Process to be Strictly Markov 


5.18. Theorem 5.9, A sufficient condition for a Markov pro- 
cess continuous from the right to be strictly Markov is that 
its transition function satisfy the following condition: 


5.18.4, Whatever the f€C , the function 
F(a, y)= fre. yi t, dz) f(z) (5.79) - 
B 
possesses the following continuity property: 


pee ere) (5.80) 


uVSs 


Proof, We write . for the class of all #@-measurable 
bounded functions in E and 5. for the class of all functions 
f€L such that 


F(u y)= f P@ yi t. d2)f(@) 
E 


is a #y,~) X #-measurable fumction of u and y, We note first 
of all that, by lemma 1,7,F(u, y) is a @-measurable function 
of y for any [Ef and any u>0. If f€C , condition (5.80) 
is satisfied in addition, and by lemma 1.10, F(u, y) isa 

#0, ; X ®-measurable function of u and y. Thus W2c. But 
o€ is obviously an -system, and by lemma 1.8,4¢2 97. We 
see on putting f=yp(€ #) that the function P(u, y;, tT) is 
measurable with respect to u, y, Condition 5.3.4 is thus 
fulfilled, In view of theorem 5,6, it remains to verify that 
5.13.4 is also fulfilled, Let f€C » and let t be a random 
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variable independent of the future and s-past such that 
<<t. Let the points ¢; define a canonical sequence of sub- 
@ivisions {Aj} of the segment [s, ¢]. We put 


Ty (w) = th, if =(w) € Ag. 


The random variable <=, takes only a finite number of values, 
Expression (5,31) is thus applicable to it, in accordance with 
the remark at the end of sec, 5,8, and 


Ms, « (f (x) fom: | =F Cn Xz,) (a.c. 2 Ps, x), 


where F(u, y) is given by (5.79). The restrictions of+, to 
Q, are clearly of2-measurable, and by lemma 5,1, (4,2, <%1€ 
of: for any A€cM. We therefore have for all n and any 
AC Ai: 

My ohahe, fC) =M, a halinct? Fu *:,) (5.81) 


We observe that tnt* for n-+oo. Obviously, 4,<: 7h <: 

=%e. Using condition 5;18.A and the fact that the process 

is continuous from the right,we have F(t, x,)%F(<, x.) + On 
passing to the limit in (5.81), we get a . 


M,, oka (x,) = M,, ata? (z, x,). 


Since the function F(x. x.) is o#f-measurable, this relation- 
ship is equivalent to (5.53). 


Remark, As is clear from the proof, the assertion of 
theorem 5,9 still holds if the transition function satisfies 
condition 5.3.4 and the following weakened variant of condi- 
tion 5.18.A. 


5.18,4°, For every f€C and w€2 , the function 
Oo, W=FUu. x) = [Plu xy 4 dz f@ 
BE 


is continuous from the right in regard to uz. 


5.19. In the stationary case it is advantageous to modify 
slightly the continuity condition 5.18,A and to replace it by 
the following: 


5.19.4, Whatever the f€C, the function 


Fi= [PQ y, dz) fe) (5.82) 
E 
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is continuous with respect to y for any 4. 


Condition 5.19.4 was first discussed by Feller, and we 
shall refer to stationary Markov processes satisfying this 
condition as Feller processes, In general, if condition 
5.19.4 is fulfilled for a stationary transition function, we 
shall speak of this as a Feller transition function, 





Theorem 5,10, A Feller process continuous from the right 
is strictly Markov. 


Proof, In view of lemma 5,9 and the remark on lemma 5,3, 
condition 5.3.4 is satisfied, By theorem 5,7, we only need 
to prove that condition 5.14.4 is also fulfilled, ‘The proof 
follows the same lines as that of theorem 5.9. 


Let f€C and let < be a random variable independent of the 
future and s-past. Let the points f{ specify a canonical 
sequence of subdivisions {a7} of the interval [s, oo), 

We put =,()=“ if t(w)=A", In accordance with the remark 
at the end of sec, 5.12, we are justified in applying expres- 
sion (5.39) tc ™, so that 

Me, 2 [f (eqn) Lofts,} = Me, f (%) =F (%,) (840422, Ps, 2) 
where F(y) is given by formla (5,82). If A€oM#?, then 
AG of, , and 
fF t0) Psa (do) = fF (x,,) Py, 2 (do). (5.83) 
A A 


As n-oot,}1, On passing to the limit in (5,83) and 
using condition 5.19.4 and the fact that the process is con- 
tinuous from the right, we get 


f (%.2n) Pe, 2(dw) = [F &z 
Jf Ps, de) [Fo Po2tdoy (5.84) 


whence (5.59) follows, 


: Remark, The above proof of theorem 5.10 remains unchanged 
if we replace the Feller condition 5,19.A by the weaker con- 
dition: 


5.19,A°. For any f€C, w€Q, the function 
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P(x)= f P(t, xp dz)f (2) 
E 


is continuous from the right with respect to t, 


5.20. We shall say in future that the Markov process ¥— 


(x, © of, Ps,2) is continuous from the right at the point x-if 
{= x} {lim x,=x}for any ¢t>06. We shall say that the 
u 


stationary Markov process X is of Feller's type at the 
point x if, for any f€C, the function F(y) given by (5.82) 
is continuous at the point x, A word for word repetition of 


the arguments used in proving theorem 5,10 leads to the 
following variant of this theorem, 


Theorem 5,10’, If a stationary Markov process is contin- 
uous from the right and of Feller's type at every point x of 
some subset G of the phase space £, condition 5.3.5 for 
being strictly Markov is satisfied for any random variable + 
such that x,€G for all w€2,. Theorem 5,9 can be similarly 
localized, 


5.21. The conditions imposed on the process in theorems &,9 
and 5,10 remain valid for any widening of the basic s~alge- 
bras, Hence if these conditions are satisfied, not only the 
process X, but any process obtained from X by a widening 
of the basic o-algebras, is strictly Markov, and the same can 
be said of any process subordinate to X (see sec. 5,15). 


We put A€oM,o if Ac, and {A.C >v}€oM for any v>t. 
Obviously, of#7,) is a c-algebra in the space Q,. 


Theorem 5,11. Let X=(x, 6, off, P.s,2) be a Markov process 
continuous from the right in the topological measurable 
space (E, @, #). let X satisfy condition 5,18.A and be a 
stationary Feller process, Then X’=(x, [ Miso Py, z)is also a 
Markov, and indeed a strictly Markov, process, 


Proof, In view of the remark just made, we only need to 
prove that X’ is a Markov process, The system X’ clearly 
satisfies conditions 2,1,4-2.15.B, so that it only remains 
to prove condition 2,1.F’, 


We start by supposing that X satisfies condition 5,168.4, 
Let fEC,0<¢s<t<cu and let vE¢, uw}. By (2.5), 


Ms, 2 {f (x,)[ of5} = FU, X) (Bale Qi. Ps, a)s 
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where 
F (vu, vy) = Mo, of (4w- 
If A€oMjso , then [4,2 >v}E€o%,, and 
Jf f@dPn2@oy)= ff Flo, x)Ps, (de). 
{42> 0} {4,5 > 0} 


Qn passing to the limit here as v|t and taking into 
account 5,186.4 and the facts that x, is continuous from the 
right and f continuous, we find for any <4: 


J F)Ps,2(do) = f FU, 2) Peo (do). (5.85) 
A A 


On letting 41+, we find that this equation is also valid 
for u=¢ . Moreover, it may easily be deduced, on the basis 
cf lemma 1.8, from the fact that (5.85) is satisfied for all 
féCc , that these relationships are satisfied for every @- 
measurable bounded function f (cf, the end of the proof of 
theorem 5.6). 


Putting f=yr , we get 
P.o(A, xyET= f P(t xy 4 T) Ps, 2 (du), 
A 


which is what we wanted to prove, 
Now let X be stationary and satisfy condition 5,19.A, Let 


fEC.UKsKt<u , In accordance with (2.5), 2.6.F and 
2.5.B, we have for every <>0 : 


M,, 2 if (Xuzs) | At.) =F (£1;.) (a,c Que Ps, x), 
where 
F(y)= Mess, wf (Sui) = Myf (%u-t)- 


Let A€oMt.o, Then {4.¢>t-+¢) Coffs, and 


Jf £us)Ps, 2 (do) = fF (xt44) Ps, 2 (do). (5.86) 
A A 


On letting :|0 , we have 
f £4) Pe, 2 (do) = f FOP. 2(da), (5,87) 
A 4 
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Just as in the first half of the proof, we can show that 
(5.87) is satisfied for all #-measurable bounded functions 
f. We have with f=yr: 


P,2(A. x,€T)= if Pit, x3 4, T)Py,2 (do), 
A 


and we have now established that condition 2,.1.F° is satis- 
fied, 


Corollary, (The law of zero or unity.) If the assumptions 
of theorem 5,11 hold, P,,2(A) is either zero or unity for 


any AC of}, ond". 
For, in accordance with theorem 2,1° and 2,1.E, we have 
Ps, o(AloMs+0) = Ps,2,(A)=Ps,2(A) (0.0.2, Py, 2)- 
On the other hand, in view of 1.6.4, 
Py, 2 (AfoMiso) = %4() (2 Qy, Py a). 
Our assertion follows on combining these two equations. 


Remark, The necessary and sufficient condition for a func- 
tion <(w) satisfying condition 5.1.A to be a random variable 
independent of the future and s-past for the process X’=(x, %, 
ohj.0, Psx) is that {o 7(w)<t<f(w)} Eom} for any t>s. 


It is also necessary and sufficient for this that, for any 
e>0 the function «t-+e be a random variable independent 
of the future and s-past for the process X=(x, % oM Pac), 
(We leave the proof of these statements to the reader), It 
seems natural to describe such functions « as random variables 
independent of the distant future and s-past (for process X), 
If xX satisfies the conditions of theorem 5,11, we can apply 
to variables independent of the distant future and s-past all 
the formulae proved in Chapter 5 for random variables 
independent of the future and s-past, 


5.22. In order to apply the criteria obtained in the present 
article for a process to be strictly Markov, we need suitable 
tests for continuity of the process from the right. These 
tests will be deduced in the next chapter. By using these 
in conjunction with theorems5,9-5.10, we shall construct in 
Chapter 6 a number of important examples of strictly Markov 
processes (see articles 4 and 7). 
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One instructive example may be considered here, however*) , 


We suppose that a particle can either move uniformly to the 
right on the half-line [0,o0o) or remain at rest at the point 
0. Further, if it is situated at the point 0 at some 
instant 1, it remains there until the instant ¢+A with a 
probability of e-* independently of the time which it has 
already spent at 0. We have here a non-cut-off stationary 
Markov process X=(x, WM}, Ps) With the transition function 


Pi, 45.0) =P(t—s, « Y= 
yrt—s+x), if x>0, 


ee (5.88) 
fer@mge (udu tyr (ent-9, if x=0. 
) 
It is clear from this that, for x«>0, 
F(a f Pb x dn fW=fe+) (5.89) 
0 


so that the process is of Feller's type for all points «>0. 


Let « be any random variable independent of the future and 
s-past, As in the proof of theorem 5,10, we construct ran- 
dom variables *,|1t satisfying condition (5,83). To prove 
that X is a strictly Markov process, we only need to show 
that (5.83) becomes (5.84) as n-+0o (assuming that /f€C). 
By (5.89), we can do this simply by proving that 


Prolt,=0, x, >0 for n=1,2,...J}=0. (5.90) 


We put 
D={o:*%,=0)}. 


Suppose that w,, o.¢€D and <(w)=—t<t(w,) . We have 


C={(D, c= t= (t=, 4,=0} EN 
and by lemma 1,5, 
xo (w) =f (xe, tere Meee ) (5.91) 


*)We leave it to the reader to give a strict statement of 


this example, i.e. to construct the space ©, the function 
*;(“) and the measures Ps, , 
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where f(x, ..-. %,.--)is a @°-measurable function in the 
space E~ and #,,...,¢,...€[s, tf]. Evidently 4+,(0,)=*%, 

(#,) =0 for all uw€[s, t], and yo()=xe(w,) by virtue of (5.91), 
But this contradicts the fact that w,€C, «EC , The contra- 
diction proves that the function 1(w) takes only one value in 
set D, If this value is equal to ¢, we have 


Hence, for any 3 0 , 
{x =0, x, >0 for n=1, 2,...)}6(x,=0, X43 > 0}. 


But in view of (2,15) and (5.88), 


Po, oc {Xp=0, %,43 > 0} = P(s, x; ¢, 0, +4, (0,0c0)) = 
= P(s, x; £, 0) PE, 0;£ +8, (0, 00)) < P(t, 0;£+8,(0,20))= (5,92) 


oO 
= f e-@-“du + 0 
0 


as 810 , which proves (5,90). 


We notice that X’=(x,, Wes, Ps,2) is also a Markov process, 
For we can deduce relationship (5.85) precisely as in the 
proof of theorem 5.11. In view of (5.88) and (5.92), (5.85) 
becomes (5,86) as «]0 . At the same time, X’ is not a 
strictly Markov process, For the instant «=§, of first 
departure of the trajectory from the set {0} is a random 
variable independent of the future for X’, We put A=@,, 
n=t+1, T={1}. The left-hand side of equation (5.8) is now 
equal to unity, whilst the right-hand side is equal to zero, 


CHAPTER 6 


CONDITIONS FOR BOUNDEDNESS AND 
CONTINUITY OF A MARKOV PROCESS 


1. Introduction 


6.1. The present chapter aims at deducing the conditions in 
which the processes corresponding to a given transition func- 
tion include at least one of the trajectories which possess 
definite properties of boundedness or continuity. 


The fundamental results will be obtained on the assumption 
that the initial transition function P(s, x; ¢, T) is normal, 
i.e, satisfies the condition P(s, x; s, E)y=1 for all s>0, 
xEE, The processes corresponding to a normal transition 
function will be described as normal, 


We shall take as our basis the following general theorem, 


Theorem 6,1, Let X=(x, 6, off, Ps,2) be a normal Markov 
process in the measurable space.(F, #) and let 7 &2z *), 
We assume the existence of the non-negative function 

GAS ty Xp sees tgs Kye ---) 


GEl0, +0]; 4... 4, ++. €[0, + 20); 
Xp veer Xys os. GE), 


which is a #°X #”-measurable function of 4. x, ..., X_-.. 
and which satisfies the following conditions for any denumer- 
able everywhere dense subset {/,, 4, ..., ty, ...} of interval 
[s, 00)} = 
61a. IGS fy Xp ees tye Xp js 
ak AC Te oa A 


if x,=x; for all indices i for which 4<, . 


*)The space 2, is defined in sec, 2,11. 
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6,1.B. If A€[s,+ co] and Gh ty x... ty dy. =O, 
definite function ¢ can be found in the interval (0, *) aie 
belonging to Y such that ¢¢)=*i for all 4,€[0, i) *). 


6.1.C. Mg, 29 Gs bys Xt ves ty Xty ...)=0**), 


Then there exists a Markov process equivalent to X such 
that all its trajectories belong to 2, 


Proof, We write E for the set got from £-by adding the 
extra point a, and @ for the s-algebra composed from all 
sets of the form B, BU {a} (BE #) - 


We put Z (0) =+ 09, 
ei | x1(w), if F< E(w), 
a, if ¢>£(o), 


P.,2(A)= Ps, x (A) (x€E, AC A"). 

We use J¢ to denote the c-algebra in the space Q genera- 
ted by the sets {w:%,(w)ET} (u€ls. 4. TE #). It still 
remains to define P,,(A). It is easily shown that every 
set ACW either does not intersect with @=(t<s}, or 


else contains ©°, We put 
(0, af An =3, 


P,, A) = a 
ae) 1, if ADQ* sxx), 





*)To fulfil condition.6.1.B with 1=s we simply require .¢ 
to contain any function with ‘[0,s) as its domain of definition, 
**)The function 7(5 tn *1,,--+ tn ty +--)Cam be regarded as 
uniquely defined for all weg, For, although the values 
of the xt, are not defined for 4,>¢, the expression ¢7(5 4, 
X4,2-++> tn: ttn ---)does not depend on ~s, for t#:>¢ by virtue of 
6.1.4. We can therefore assign arbitrary values to x, for 
t,>% Without their choice being reflected in the value of 

AG ty, 45-06 tas Xtgs eds 
***)This definition is incorrect if 9:9, since for every 
Aced* we now have simultaneously A>2* and 4p2*=g. But we 
can suppose without loss of generality that ¢()=0for some 
m€2, so that G is non-empty for any s. For, if there is 
no element ») with this property in the space 9, we can 
enlarge 2 with the single point » and put (m)=0, P, .({#9}) 
=0 (s>0,x€£), whilst retaining the previous values for the 
symbols x;(#) and o#%, It is clear that, with this, a Markov 
process is obtained which is equivalent to the initial pro- 
cess X and which satisfies, along with xX, the conditions of 
theorem 6.1, 
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We have for any 0<s<t, x€E, TER? 


Pi, x tT) =P, 2 x,EP} = 
_{P@ 64 TNA) +y@l~— P@ hit x#a, 
(1%) if x=. 

The system (x, 7. oi. P.,2) may readily be seen to satisfy all 
the conditions 2,1,A-2,1.E and 2.1.F°, so that it defines a 
(non-cut-off) Markov process X, (When proving property 
2.1,F’ we have to use the relationship 4\?[2,J]—WN%.) 


We define the mapping 2:2; as follows: if 9() ¢€[0, 
)) is a function of Q¢, we put 


g(t) for t€{0, d), 
«=| % for €[P.+00). 


We bring in the following notation: 


P=uUTLf); == inf (ty: %_ =a}; 


l, if #2 hand x,€E, 

DF eibge He oS nae . 
ily * Be i in remaining cases; 
7 (bp Xp sees Ege pp =a ty Xp vere bas Xgr eee 


+ 2? (fy, Xp. oe 0s bye Xgv ---)e 


The function ¢ is clearly measurable with respect to x, ... 
Xess Let tty eer tas ---$ Sls, oo). TE Gb Xp ees ba Mae oe) 
=0 then (454, x, ....t, %_..--)=0 and *%=2 for all #,>72. 
In view of 6,1.B, there is a function 7 of .Y, defined in the 
interval [o, {) and such that e(f)=—-x,; for all 4<7, The 
function» =ap€ _Yevidently satisfies ¢¢)=~x; for all i, 
Condition 2,13,A is thus fulfilled for the process X and the 
space #, Moreover, for all wE€Q, 


K(ty Xt --2s Ege 
MD Asean Bek eas 

Therefore 
a(t» Mivoe scb as Xtyeo- jJ=7 fr, Xt, 00s bys Xe -) 


and by 6.1.C, condition 2.13.B is fulfilled, By theorem 2.6, 
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a process xX’ equivalent to X exists such that all its trajec- 
tories are contained in Y, The set £ is obviously inacces~ 
sible for this process, In view of the normality of process 
X , the set £ satisfies condition 3.8.B with respect to the 
process X’ , By theorem 3,4, we can form a part X” of pro- 
cess X on the set E, It can readily be seen that X” has the 
same transition function as the initial process X, and all 
the trajectories of X” belong to , The theorem is proved. 


Remark, Let all the trajectories of process X belong to 
the set %)G@, and let the function 9()j ty x, .... ty Xa -.-) 
satisfy conditions 6.1.4, 6.1.0 and the following condition: 


6.1.B°,. If s@¢E[0,)))is an element of space P and 
GAS by BU) oo es Far Pn) ---) = 


there exists a function 9€.9fN.% defined in the interval 
(0,4) such that 9(4,)=¢¢) for all 4,€[0, i). 


Then there exists a Markov process equivalent to X such 
that all its trajectories belong to 9Nfo, 


Proof, We take the process X, the mapping «:Q,-> Bl) and 
the function 7 constructed in the proof of theorem 6,1, Con~ 
ditions 2,13. Ae and 2,13.B are fulfilled for the process x ; 
the function q and the spaces P=a(P), Ly=2fr). Hence, 
in accordance with the remark on theorem 2,8, a Markov pro- 
cess xX’ equivalent to x exists, all the trajectories of 
which belong to $n Fo - The part X” of this process on the 
(inadmissible) set EF is equivalent to X, and all the trajec- 
tories of X” belong to $N.Y. 


2, Conditions for Boundedness 


6.2. Let ¥ be an arbitrary system of subsets of the set 
E. We shall describe the function 9(4(#€[0,A)) with values 
from E as ¥-bounded if, for any T<}, there exists TC¥ 
such that 9(€IP for all t€{0, T], ‘The Markov process X= (x,, 
Ai, P, <) in the measurable space (FE, #) will be described as 
F bounded if all its trajectories are &¥ -bounded, 


We shall assume that the system ¥ is subject to the follow- 
ing condition: 


6.2.4, A sequence I,\EI,¢...E17,&... exists, consisting 
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of elements of the s-algebra # and equivalent to ¥ (see 
sec, 3,11)*). 

It may be remarked that a Markov process X is oF -bounded 
when and only when t=—'(#) (the function.t(@#) is defined by 
forma (3,58)). If the system ¥ satisfies condition 6,2.A, 
this equation can be written as follows: 


| ene Fi 4 
C= tim $ 0s) (6,1) 


(the function $1) is given by (3.49)). 


Let T€ g and let A be a subset of the interval [0, 00). 
We put 
%O=[\lerh ©) =U {x€ ENT}. (6.2) 


tea 


The sets ¥,(T) and ®,(1) are obviously disjoint. For a non- 
cut-off process their sum forms the space 2, This is not 
generally true for a cut-off process, 


Lemma 6,1. If X is a Markov process in the space (E, #), 
TE gw, and.A is a finite or denumerable subset of the inter- 
val |s, 4], containing the right-hand end ), we have for any 
GEF: 

Palla MSPs. x; 6,G)— sup P(u, y; 6, G). 
YET, UEA (6.3) 

Proof, We suppose first that the set A is finite, and 
put 


eee if oé¥, 7), 
oe b, it wE€¥, (1). 


Obviously, < is a random variable independent of the future 
and s-past (2,=®,()u¥,(%). Lemma 5,4 justifies us in 
applying expression (5.8), so that 


*)This condition is satisfied by all normal systems and in 


particular, by systems 3,12,1-3.12.4 (see sec. 3,12). 
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P(s, x; 6, G)=P, 2 {2., x) EG} = f Po x; 6, G)P, 2 (dw) = 
* (6.4) 


= f P(t, x,; b, G)Ps, a(dw) + f P(s, x.; b, G)P,, 2 (dw). 
¥,C) 4) 


Expression (6.3) follows from (6.4), 


If A is denumerable, we consider the sequence of finite 
sets A, tA(we choose the A, here so that they contain 4), We 
can apply (6.3) to each set A,, and on passing to the limit 
we find that (6.3) is applicable to the set A, 


6.3. Theorem 6,2. let the system ¥ ]# in the space (E, 9) 
satisfy condition 6,2.A and let the normal Markov process 
X= (Xp 5: offi, Py, a) satisfy the following condition: 


6.3.4. There exists ’>0 and a sequence of sets E,€ # 
such that £, f£& and for any a: 


inf su P(u, y; 0, E,)=0. 
foie (6.5) 
O<Sv— usd 


Then an ¥-bounded Markov process exists, equivalent to X, 


‘Proof. We write £& for the set of all ¥-bounded functions, 
By theorem 6.1, the problem amounts to constructing a func- 
tion q(i; ty, x, ...5 ty Xp .-.)Satisfying conditions 6,1.A-6.1.C. 


We take the sequence of Lf, which is equivalent to # and 
for which condition 6,2,4 is satisfied, and introduce the 
following notations 

h(x) == sup {k:x ET} *), 
hes bys Kp ones Egy Xqe ee DS 
sup {A(x,):t, <<A—e} for A\<o, 


\ sup{ h (xp) te <=} for A=-+0oo. 


We put qQs ty Xp, 2-0. tyr Sq oe +) OAL AA ty Xp... by 
Xy,-..)<co for alle, If the opposite is the case we put 
G05 ty 4 ..-. ty, X,)==1. This function is clearly measurable 
with respect to 1, x; ....%,,.-. and satisfies condition 6.1.4. 


*)We put A(x)=0 if x€T, for alle. 
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let A=‘t,.....4,..-} be amy sequence of numbers and ¥*,,..., 
Xqv ee. any sequence of points of —. We consider the function 
g(t) (@EfO, 4) given by 


Xi if t=t, t<d, 


yo if t€ ATS), (6.6) 


20={ 


where y, is an arbitrary fixed point of [,. It may easily 
be seen that 96.7 if 9) ty x, ....t. Xp.) == 0. Condition 
6.1.B is thus fulfilled, 


We now show that condition 6,1,C is satisfied, Let A={t, 
... ty »--}be an everywhere dense subset of the interval 
Is, co). We put for brevity g°=4(6; ty. Xt. ---+tn ty -..), and 
define #° similarly, let « be any paella number, We 
take a sequence tending to -+-oot 4.<m<... Cyc... EA, 


such that each of the numbers y4,—s, u,—wu, ..., Une, — Up « 
is less than ¢/2, We put 
A, ={[s, uy), A, = [uy Ug], oes Ay = [4y-1, uy], ... 
Ay = AA, 
y= Ta, (E) = {f > uy}, Te) = Ya Tn): 
It will be observed that 
{Aj = co} «U A) AER}. (6.7) 
mal 
In view of lemma 6,1, we have for each E&, : 
Ps, 2 (¥E") > Pls, %; Uy E,)— _ sup Pa, y; uy E,). 
YET yy UEA, 
If 5<2, it follows from (6.5) that 


lim sup P(4, y; uy E,)=0 
mroyer., ucAz Yi ty En) c 


Hence for any z, 


lim Ps, o(¥E"} > Ps. x; ty, Ep) 


mM >co 


and therefore 
oa Ps, z (UNEP j= as Pez a {¥x} , Le nes z {UP} < 


6.8 
<P(s, x; m, E)— ay 4 x; W,,E)=0. ( ) 
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It follows from (6.7) and (6.8) that 
Ps, 2 {hj = 00} = 0, 


& 


provided that <é. It remains to notice that 


v9] 


so that 
Ps,2{g> 0) < lim Py, (41 = co} =0. 
n>o 7 


3, Conditions for Continuity from the Right and 


Absence of Discontinuities of the 
Second Kind 


6.4. Let X=(x, 6, of, Ps,2) be a Markov process in the topo- 
logical measurable space (F, @, #) and let y(w) be any function 
subject to the inequality »<¢. We shall say that the pro- 
cess X has no discontinuities of the second kind up to the 


instant 7 if the limits ~x,,)() aa Yu) and. x;_9(w) = dim #a (©). 
Ese At 


exist for any w€2, t€[0, n(w)). To. say that "the process xX 
has no discontinuities of the second kind" is the same as 
saying that "X has no discontinuities of the second kind up 
to the instant ¢", Let Tf be any subset of E and ¥ any 
system of subsets of FE, We define &(I) by expression (3.49) 
and §(#) by (3.58). If the process X has no discontimities 
of the second kind up to the instant [E(M)E@?)), we say that 
it has no discontinuities of the second kind up to de e 
from T (from F. a Analogous meanings attach to the state- 
ments: "process X is continuous from the right up to the 
instant 7", "process X is continuous up to the instant 7", 
"process X is continuous from the right up to departure from 
I", and so forth, 


6.5. Lemma 6,2. If the Markov process X is continuous 


from the right in the topological measurable space (£, @, #); 
we have for any UC @ Mand any x€U, t>0: 


. jim Pi, x3 th, T)=O0. (6.9) 
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Proof. If a”, 10, we have 


; U (x=. Spin, CU} =o 
m=ln=" 


Consequently, as m—->oco , 


Py, 4 U lem € un} 0 


n=m 


But 


Py, 2{*t+1m €U} < Pac {U U Xr € ah. 


n=m 


Therefore Py, 2 {[*tzim €U}] =P, x; t-+h,, U) also tends to zero, 
which proves the lemma, 


6.6. We shall now suppose that the process is given in the 
metric measurable space (E, #,9), the distance p(x, y) being 
# X S-neasurable*), It follows from this condition (see 
lemma 1,4) that p(x, y)is a #-measurable function of y for 
any fixed x, The c-algebra # therefore contains the e- 
neighbourhood U,(x) of any point <, 

We put 


 teseeuae © x; t, U,(x)) (6.10) 


(here [TcF,«,t>0). 
We bring in the following condition, 
The M(t) condition, For any e>0, 


. ae acl aegis (6,11) 


Our main purpose is to prove the following theorem, 


Theorem 6,3. Let (E.p, #) be a complete metric measurable 


*)In accordance with sec, 18, pix, y) is always a continuous 
function of the pair (x,y). Hence p(x, y) is 9x F- 
measurable if @ contains all open sets of the space (E, p) 
{see sec, 1.9), 
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space, the function.p(x. y) being # X #-measurable, Let ¥ 
be a system of subsets of E satisfying condition 6,2,A, If 
a normal Markov process in the space (E, 5, #) is subject to 

the M(f) condition for every set TE ¥, an equivalent of this 
process exists which is continuous from the right and has no 
discontinuities of the second kind up to departure from ¥, 


6.7. We shall first of all prove a few lemmas, 


Lemma 6,3. Let X=(x, 6, of, Ps.) be a Markov process in 
the metric measurable space (E,p, g). lLetp(x, y)bea ®XF 
-measurable function, Let Té@. Then for any «>0,0<s 
gt<gu : 


P, {EP p(X Xy) pe} Sap (u—f). (6,12) 


Furthermore, if A is any given finite or denumerable subset 
of the interval [a, b] (s<a<b) and 


D=Di(A)= U (x€T. HEP. x.ET, plxp %,)>4e} (6,13) 


tues 


we have 


Ps, 2(D) < 2a8 (6b —a). (6.14) 
Proof, By (2.9), 


Poo (MT. (ap x) >} = f Py, 2, { (Xp X,)>¢} Pp,» (du) = 
ver 


g f PU, x5 2, Ti CEDIPs, »(do)<at(u—d), 
@,€r 


and expression (6.12) is proved, 


We turn to the proof of (6.14). First, let the set A be 
finite and consist of the points t<t<...<t, . We write 
« for the least among the numbers /€A, satisfying the con- 
ditions x,€T, p(x. x,)>2e (if there are no such numbers, we 
put t= max(s,C)). We put 


B={x€T p(%a %)>e}, C= (p(t x) De}. 
Clearly, D=Byc, so that 
Ps, 2(D) < Ps, 2(B)+Ps, 2 (C)- (6.15) 
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The random variable +: is independent of the future and s- 
past, 


We apply the corollary of theorem 5.1 to the quantities 
<(0), %)=t) % (0) =FWE Q.) and to the function 


=| 1, if pO y)2e 
fOv J) = 0, if ep Ye) <e 


(in accordance with the remark at the end of sec, 5.9, this 


is justifiable because t. y, and ™% take only a finite number 
of values in 2.), We have 


P,, 2 (Clo&:) = Mg, z {f Xa» Xx) | oft:} =F x Me a) = 
=F (ct, %13 % 5) (2.0 2,. Ps, 2)» 


where 
F(u. Ys Uy %)= May, yf (Xo. %0,) = Pu, y (P(%o, Xo.) >}, 
so that 
F(u, y; 2, 1) =P(u, y; b, T. Gy). 
Obviously, F(u. y; uw, b)<as(b--a) for any u€[a, b], yer. 
Consequently 
Ps, 2(CloMt)<ak(b—a) @.0.2., Py, 2) 


and 
Ps, 2(C)< at (6 —a). 


Moreover, in view of (6.12), Ps,2(8)<o,(6—a). On substi- 
tuting these inequalities in (6.15), we get (6.14). 


If A is demumerable, we can choose a sequence of finite 
sets A,tA. Then Di(A,)+Df(A), and we get inequality (6,14) 
for the set A by a passage to the limit, 


Lemma 6,4. Let X be a Markov process in the metric 
measurable space (E, p.- #) with a #X @-measurable function 
p(x. y) . Let Té€g, and let A be a finite or denumerable 
subset of the segment [a, b] S[s, co). We put 
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A, = Ai, (A, T) == {there exist s5< <1. SSH < 
<% of A such that x, €T. x, €T, P(g» Xey,) > (6,16) 


>4e for i=1, 2,..., R}. 


Then 
Ps, 2 (Ax) < [2ah (b—a)]*. (6.17) 


Proof. We suppose first that the set A is finite and 
consists of the points 4<t4<...<t, . We take the random 
variable % , equal to max (s, 0) outside A, and given in A, 
by the equation 


ty (0) = inf {tn 2 OE Abt, -. 6s bail}. 
We put 


4 = Max (ty, T%) @=90, 1,....m). 
Obviously, 
n 
fe {%,€P, *,€T, a, ET p (x4,, %n,) > 4c}. 
The characteristic function of A,,, is therefore equal to 
Ff (%qo ++ ++ %q,,) 9 Where 


{1. if y,€T and there exist ¥ ET, 


fOp -+0) I) = for which p(y. ¥,)>4e, 
0 otherwise, 


The quantities t, are independent of the future and s- 
past, whilst the 7...,%, are ofM:,-measurable, We have by 
the corollary of theorem 5,1,: 


Ps, a (Axs1]o%,) =F (th Xa5 Nh oees 7.) (A.C. 2,,.P 8, a) (6,18) 


where 
P (ty Yi Op ++ +1 Un) = Mu, yf So +++ %0,) = 
= Py, y[Dp (Or -- +) Undl- 
We conclude from lemma 6,3 that 


FQ, Y; UW»... 04) < Lap (V, — 03) < Lap (b —a). (6.19) 
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From (6,18) and (6,19): 


Py, z (Ax1) ard Py, a2 (ApAga1) = (6 ) 
220 
= a Ps, 2 (Anas oM?,) Py, 2 (do) < 2ah(>—a)P,, (Ay). 
k 


Expression (6,17) follows readily from (6.20). 


If A is a denumerable set, we choose a sequence of finite 
sets A, tA and observe that Ai (A,, T)t Ar(A, T). A passage 
to the limit shows that (6.17) likewise holds in this case, 


Lemma 6,5. let the Markov process X=(x, 6, oM2, P,,,.) in the 
space (F, @, @) satisfy the:.M(I) condition T€#), If «,EPr 
for all ¢€[0, 7) and process X has no discontinuities of the 
second kind up to the instant 7, a Markov process X equiva- 
lent to X exists which has no discontinuities of the second 
kind and which is continuous from the right up to the instant 
1. 


Proof, We put 
F = (Re when ! <7, 
: x, when 7<t<’. 
We have for any s<z: 
Wet = (>t x # x40} S (ET. x, % X40}- (6,21) 


We notice that, if e>0.s<¢t<u, then by theorem 2,1 and 
2.1.D, 


Pao (HET, pet x >) —= f Pr, o, (ol%n x,)>>¢} Py,» (do)= 


wer 
— ai Pd, x; 4, U, (x) Ps, 2 (dw), 
2,¢r 


and in view of the M() condition: 
nae Psa {ET p(xp xy) >} =0. 
But p(x, xy) p(Xp X49) 88 ult , so that 
er x.0>) SUN {f (* Ah) = 


n=lk=n 
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Hence 
P,, F) {+E Py p(X X40) > s} < 


= let Pac {ne Tr, A [? (te gr > ‘} < 


< jim Pea {a€P. P(e *,42) a0 


Since this is true for any <«>0, we have 
Ps, o{%r:EP, x,#x¢49} =0 


and in view of (6.21), Ps2{y#x,}=0 . An obvious conse- 
quence of this relationship is that X= (vp 6. P, 2) (the 
s-algebras Wz are defined in sec, 2,2) is a Markov process 
equivalent to process X,. Evidently, % has no discontinui- 
ties of the second kind and is continuous from the right up 
to the instant 7. 


6.8. Proof of theorem 6.3, In view of lemma 6.5, we sim- 
ply need to construct a process X equivalent to X and 
having no discontinuities of the second kind up to departure 
from #, We make use of theorem 6.1 for this construction, 


Let 9(4) be a function in the interval [0, \) with values 
from £.. We put 


8. 9) inf {t: PET} | 
CF. e)= supth 9) | (6.22) 
Tex 


(if PEL for all ¢€[0, 4), we put ET, g)=)). We shall con- 
sider » belonging to the system . if the limits ¢(f+0) 
and g(f—0) exist for each /€[0. §(¥. 9)). 


Let e>0, A= {ty .... tas ---} 10, Co), Hy... Xe. CF , 
and let A be some interval, We put 


inf{t,:t, ch, x, EP}, 1f tn<d, x,€T 
aw =| {fat fn < oT) for some fn, (6.23) 
i in the remaining cases, 


__ = a 
E=h (F)= us ET), (6,24) 
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1, af x€Tandp(x, y)>4e, 
hi Y=) otherwise 


and we define &,sQs ty %p --+> tn: Xp ---) aS the upper bound of 
the values of the sum 


A, (%i xy) +h, (is: ig) + aes + h, (Fieg—1 Xion); 


when n runs through all natural numbers, the indices4, ..., i, 
being only subject to the condition that t,<f,<...<4,, 
E€AnfO.&) *), We put gtx, ota te J=0 If mAh 
Xp... ty Xp -..)<0o for any e>Q and any finite interval A, 
Otherwise, we put 7Q; ty, xy ~... ty Xp ---)=1. We note that 
the function g is measurable with respect tu \. %p).-., 4%... 
This follows from the fact that A,,, depends monotonically 
on e and A, and we can therefore consider only « and A run- 
ning through certain denumerable sequences when defining g¢, 


The function ¢ clearly satisfies condition 6.1.4, We shall 
verify that it also satisfies conditions 6,1,B-6.1.C, Let 
A={t, ..., fy ...} be an everywhere dense subset of the inter- 
val fs, oo), If gts t, x ..., ty Xq...)==0, Whatever the 
monotonic bounded subsequence 4, ti, ....4,,...<& of the 
sequence %,..+, f,, .... the corresponding sequence of points 
Xi,» +++. %,,-.- Satisfies the condition: for any «>Q there 
exists n(e) such that P(*iq %i,)<e¢ for m,n>n() . In view 
of the completeness of space E, the sequence Xiyreeen Kigy oes 
is convergent. 


We define the function ¢(t) (¢€[0, )) by the formlae 


Xj, if t=4,€AN[0, 2), 


eO=j, im 2 af t€ls, nA, 


e()) if f€10. 5) or if f€fk, NA. 


(6.25) 


It is easily seen that &(F,g)<t, and ift.....%....is any 
monotonic bounded sequence of points of [0, §) » the limit 
isis ? (2) exists, The function » therefore belongs to the 


class .Y, Condition 6,1.B is satisfied, 


*)If such indices can not be found, we put hy 0 
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We now show that qg satisfies condition 6,1.C, Let aso, 


A=[t,..0 ty ...JE[s, co). We put 
+ _.. i“ 
q =9(5 hy Xt ceas ty Xt +e), 
* 
Wehbe Kt wes by Ftp +++), 


An=[s-- m8, s+(m+-1)3] (m=0, 1, 2,...), 
Clearly, ae 
{9 =U Ue , ng (6.26) 


n’' ™ 


let TrS...¢7,¢... be a sequence of sets as described in 
condition 6,2.4, Then eT) t§ and therefore, for any e>0 
and any k, 


1% a, = LSU And TD. 
where A; is defined by expression (6.16). It follows from 
this that 
P,, 2 { he oer oo} Slims, 2 {4.can An T)}. 


By lemma 6.4, 


P,, z {Ay (An An, Tr} < [2a @}*. 
Consequently, 
Ps, 2 {Ms = 00} < [2a @]. 

By the hypothesis of the theorem, for any e>0 there 
exists a @>0 such that of(3)<1. We have with this choice 
of 8, - 

Pro{M », =} <(a)" 
and, since this is true for any &, 
Ps, = { A, 4, 00} = 0. 


On combining this equation with inclusion (6.26), it may 
be seen that P,,.(7*>0)=0 , and the theorem is proved, 


6.9. Remark 1, Let (E£. p, #) be a metric measurable space 
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satisfying the conditions of theorem 6,3, and let ¥ bea 
system of subsets of EB equivalent to a sequence of closed 
sets L.Ghne...SIr,¢G.... If the normal Markov process x 
in the space (EF, p, #) is #-bounded and satisfies the M(I) 
condition for all Té gg, a Markov process equivalent to xX 
exists which is #-bounded, continuous from the right and 
devoid of discontinuities of the second kind, 


Proof, We define & as in the proof of theorem 6,3, and 
write Lo for the class of all ¥ -bounded functions of Q,, 
We notice that, if $€.% and x,=— )(¢,), the function 9(t) given 
by (6.25) belongs to ¥N.%,. It may be concluded from this, 
in view of the remark on theorem 6,1, that a process X 
equivalent to X exists, all the trajectories of which belong 
to £NFo. We have §{(¥)—* for this process, which proves 
our statement. 


Remark 2, Let (£, p) be ac-compact complete metric space, 
# the c-algebra generated by all open sets of (E, p) and P(s, 
x;#,T)the normal transition function in the space (F,p, g) . 
We assume the existence of a sequence of closed sets T, tE& 
such thats 


a) the M (f,) condition is fulfilled for each 1, 3 
b) there exists 0<@ such that, for all m, 


lim su \° —_ 
gee Pia, y; uv, T,)=0. 
ver, 
lo-u, <é 


Then there is a Markov process with the transition function 
P(s, x; #,T) which is {I,,! -bounded, continuous from the right, 
and devoid of discontinuities of the second kind*), 


If condition 5.18.4 is satisfied, or if P(s, x;4,.)=P¢—s, 
x, T), where P(t, x, T)satisfies requirement 5.19.4, this pro- 
cess is strictly Markov, 


Proof, By theorem 4.2, a Markov process X exists with the 
transition function Pis, x; #,T). We put & =({T,}. By theo- 
rem 6,2, we can construct an ¥-bounded Markov process 
X’ equivalent to X. This process satisfies the M(0,,) condi- 
tion for every n, In accordance with remark 1, a process X 


*)Needless to say, it is sufficient to require that the M(4) 


condition be fulfilled instead of condition a)-b). 
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equivalent to X’ exists which is ¥-bounded and continuous 
from the right and which has no discontinuities of the second 
kind, When condition 5.18.4 or 5,19,4 holds, this process is 
strictly Markov by theorems 5,9 and 5,10, 


4, Jump-Type and Step Processes 


6.10. The Markov process X=(x, % off, P, 2), in the measura- 
ble space (£, @#)is said to be a jump-type process if, for 
each w€Q, 4€{0, C(w)) there exists > 9 such that 71, (0) =x, (w) 
for all O<A#<8, The number ¢ is called the instant of 
jump of the trajectory x,(w) if the sequence ¢,+¢ exists such 
that x1,() #x,(o) (n= 1, 2,...), The instants of jump may 
easily be seen to form a denumerable set, If, for any wo, 
this set has no limit points inside the interval |0, C()) , 
we shall describe X as a step process, 


We introduce a discrete topology in the space (F, #), tak- 
ing for C the class of all subsets of £, The class of jump- 
type processes in (E, #) is easily seen to coincide with the 
class of processes continuous from the right in (£. @, #)3 
the class of step processes coincides with the class of pro- 
cesses continuous from the right and devoid of discontinuit- 
ies of the second kind, It follows from theorem 5,10 that 


every stationary jump-type Markov process is strictly Markov. 


The topological space (£, @) is metrizable, The correspond- 
ing metric is given by 
0, if x=y, 


a y=, if xy 


The following is a sufficient condition for the function 


p(x, y) to be measurable with respect to #X #: 


6.10.4, The subset of space EXE consisting of points of 
the type (x, x) (x€E) belongs to the c-algebra #X#. 


The following theorem is obtained by applying the results 
of article 3, 


Theorem 6,4. Let (E, #) be a measurable space satisfying 
condition 6,10.A, and let X be a Markov process in (E. #) 
subject to the condition: 


6.10.B, The convergence is uniform over t>0, «CE: 
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lim P(t, x; t kh, E == 0, 
lim P(e, x5 t+, B\ x)= 0 


Then a step process exists which is the stochastic equivalent 
of process X, 


We suggest that the reader define a process which is jump- 
type (or is a step process) until departure from the system 
F » and state the analogue of theorem 6.4 for such processes, 


6.11. Examples, 6.11.1. Let E=(1, 2,..., m,...} and let # 
be the system of all subsets of &£. In accordance with 
4.2.2, every transition function in the space (FE, #) can be 
specified with the aid of a system of functions Pis(. DO< 
s<t). Condition 6,10.B is expressed in terms of the Diz (s, 2) 
as 


mange PT O—O- ie 


In the stationary case, this amounts to requiring the uniform 
convergence (in regard to i)of: 


lim .;(h) = 
lim 2 Pig (2) =0. 

Furthermore, if the set F is finite, the uniformity require- 
ment is satisfied automatically, and condition 6,10.B is 
equivalent to the requirement that p,;(4)>0 as A\O for any 
ixj e 


We can conclude, on taking into account the fact that the 
conditions of theorem 4,2 and condition 6,10.A are fulfilled 


in the present case, that there is a corresponding step pro- 
cess for every transition function subject to condition 
(6,26°) in the finite or denumerable set £. 

6.11.2, The Poisson transition function (example 4.2.6) is 
readily seen to be complete and to satisfy condition 6,10.B,. 
There will therefore be a non-cut-off step process correspond- 
ing to the Poisson transition function, if the phase space is 
subject to condition 6,10.A and a topology satisfying the 


requirements of theorem 4.2 can be introduced into it. We 
shall term this a Poisson process, 
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5. Continuity Conditions 


6.12. The N(f condition formulated below will be shown to 
play the same role in investigating the continuity of a 
Markov process as is played by the M(1) condition in criteria 
for continuity from the right and absence of discontinuities 
of the second kind, 


The N(M) condition, For any e>0, 
ap (8) 


lim 3 0, 
sy 
We shall make use of the following lemma when deducing 
criteria for continuity, 


Lemma 6,6. Let X:=(x, (. off, Ps.) be a Markov process in 
the metric measurable space (E. p, #) (witha #X #-measura- 
ble function p), let the set T€M, and let the subset A of 
the segment [s. 7] be finite or denumerable, We write §={ 
for the strict lower bound of elements f of set A for which 
x,éT, and put 


Q=O(M= Yo ft<h uch ple, x) >4e}. (6.27) 





be 
Then 
in (23 
Ps, 2(Q)<2(T—s FO. (6,28) 


Proof, We choose the points 4<@%<...<4,€[s, T] such 
that 


u,— SS h— hy =... Sy — y= 8, OS T—a, <8, (6.29) 


and put 


Ao==[s. ua], Ay = [ay Wg], Ay = [p, ag], s+, Ay = [ayy T], 
D=Dp(ANA) G=1L.... k—D), 


where the event Di is given by expression (6.13). Clearly, 
es D;° Therefore 


tml 


k-1 
Py, 2(@< Di Ps, 2(Di). (6,30) 
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Since each of the sets ANA; is situated on a segment of 
length 23, we have by lemma 6,3: 


Ps, (Dj) < 2ap (28), 
and in view of (6,30): 
Ps, 2(Q) < 2(k — 1) ap (28). (6.31) 
T—s 


But, by (6.29), R<T—s, Therefore p—1 <k< =, 
and (6. 28) follows from (6,31). 





6.13. Theorem 6,5. Let (E. p. #) be a complete metric measur- 
able space with # X #-measurable metric p, Let T bea 
measurable set in the space (E. p, #)and X a normal Markov 
process subject to the N(T) condition, Then a Markov process 
equivalent to X can be constructed which is continuous up to 
departure from I and is such that, for every w€Q, either 
&(T, wo) = 2 () or lim x,(w) exists and belongs to the boundary 
T of the set r*t™*) 


Proof, Let ¢(t) be a function in the interval [0,) with 
values from £, We define the quantity &(9)=§&(I. ») by expres- 
sion (6.22) and we shall say that the function 9 belongs to 
the system ¥ if: 


a,) o(t) is continuous in the interval [0,&(¢)) ; 
ay) either §(»)=2, or Bs o(f) exists and belongs to I”. 
Let T>0,8>0,A={t, .. tar «+ +} S10, 00), Xp, 2. Hm EL 


We, define the function rh ie expression (6.23) and put 
7? — min GT) 3 


By 05 bas te oo os bar Fas ane Sup a P (Xis 4) *); 
le ul <2 
he = lim hy, b= lim be, 
£0524 =| Bee ae iene yer: 
-+oo in che remaining cases; 


*)IF the set for which the upper bound is taken is empty, 


we put 
he, ity Xp .--s thy Xn, .-.) = 0. 
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8505 fy Xp wees f. xy. = : A 
=I neue ee Xt; 4), If F< 
( 0 if f=) 
BQS5 by Xue oer ba Xa oe = lim g (sedi Riis oy bie Rig lece), 
The function g=h-+g satisfies condition 6.1,A and is 
measurable with respect to i. x1,..-. Xa --- We prove that it 


also satisfies conditions 6,.1,B-6.1.C, 


Let A= (tf, .-., ty ---} be an everywhere dense subset of the 
interval [s, co), E(s, 00) and let g(Qh ty tp +++ ta tg ce.) =U 
We suppose first that =<} 3 we choose TE€(& 4). It follows. 
from the equation ho(h; t, x1, ..-) tay Xp ---)==0 that, for any 
e>0, there is a §>0 such that p(x, x))<e for t,,t,< at 
and |f4;—t;|<6,, This means that the function o given by 
¢(t;) =x; in the set Aff{s,§) is uniformly continuous in this 
set and may therefore be extended (uniquely) to the contin- 
uous function ¢(f) in [s,&), which has a limit as ttt. We 
complete the definition of this function by putting 


{o(s) for O0<t<s, 
o@)=) x, if t=4,€15.2), 
| Xx if t€(& ANS. 


The function o(t) satisfies the condition ¢(f))=~; for all 
t<h , and condition a). Moreover, (2) <— and condition 


ay) is obviously satisfied if §&(9)<&, On the other hand, 


if &(~) =&, it follows from the relationship g(\; ty %1-+++ bys 
Xp...) =0 bat at 2 PO T)=0 for any §>0. We put 
z=lime() | Byidently, p(z, F)== p(2.T)=0, Consequently 
z€I”, This proves, in fact, that the function » satisfies 
condition ay) and therefore belongs to #, 


We now suppose that §—). Since 
Aes ty Xy veer bys Xps ..-) == 0 


for all 7, the function ¢, defined in the set A by the 
expression o(f,))=—x; is uniformly continuous in the intersec- 
tion of this set with any segment [s, Tic[s,4). The function 
may be extended uniquely to a continuous function in [s,) . 
On extending it to the interval [0,s) in accordance with the 
expression 9(t)=9¢(s)(¢E[0, s)) , we evidently get a function of 
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Turning to the proof of condition 6,1,C. we write for 
brevity t& =H (1), gt—= (3 te tte +++ fe tq ‘++) and similarly 
define 4°, hr, h., 7 and q - 


Let A(t, ..., ty, --+-} be an everywhere dense subset of the 
interval [s,0o), It will be observed that, for any positive 
36,7 


{hr> 4e} 6 (ti r> 42} SCQF Ans. TI}, 
where Qi" is defined by expression (6.27), It follows from 
this, in accordance with lemma 6,6, that 
ah la 





Py 2 {Ar > 4}< 2(T—s s) 


If we let 3-+0, we have from the N S condition: P, ,(h> 
> 4c) = 0. 


Since kpth as Ttoc , we have 
co ow 
* 1 
he >— ’ 
9500 {ae>4} 
whence it is clear that P, .(h* >> 0)=0. 


We now represent the set A as the sum of finite sets 
At= {ur< tee ads <n } such that 


b,) NeMe...cA°c...; 


3, 
by) cuba <d, (R= 12... m,—1) 2 MHEFE 8,10 28 
>We 


We agree to write for brevity x? instead of <n ‘. 
We bring in the random variables: 

ta sup {ae:xtET for j= 1,2, ves Bhs 

vq = inf (4g: xR ET}. 
We notice that, for any T>0, 


{e (Aq? x, )2* ta< TS U {=P eT. ol xp. 221) > e}. (6, 32) 
wor 
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But in view of (6,12) and b,), 
Pe, 2 {2p ET, p (2p, xP.) 2&8} Sak (aR up) <3, (8 n)* (6.33) 


On the other hand, by b ); the number of points cf sets A” 
located in the segment [s, T] does not exceed *o—) gS) te There- 
fore we have from (6.32) and (6,33): 


at, (3, 
Pa, {P(%tq® Hq) 2 Pn < TL SCL, (6.34) 
where C—3(T—s)-+2, « 


We now write =, for the greatest of the elements of set A” 
less than *, It is evident that z%<p,< &<v, and v1 %% 
as n>co, On the other hand, x, ft by virtue of b o)e There- 


fore ‘n—™J0, and all the more, p,—7,—-0., We tigeiee that, 
for any T>0,e e 0,8>0 and any sufficiently large .n: 


{g* > 5e} & [F< (He, X,,) > Se} S 
(TSE < 00} U ln — a > 3} U 
U{E <7, Pn — Fn <3, (Xe,> %,) > Se] S 
S{(T<B <0} U (pn — ma > 8] U 
UQE(A* MIs. T))U[E < Tr (Xa Haq) > ef. 


This gives us, using lemma 6,6 and inequality (6.34): 
a {8 > 52} < Paya (TCE < 00) + Pa lm— a> 2) + 


+279 2 


If we first let 2 tend to infinity, then 8 to zero, or 
eer T to infinity, we now get P, .{g*>5e}—0. Thus 

P.2{g°>0}=0. The relationships proved above: P, ,. {h*>0} 
=P,,2{g* > 0}=0 show us that P, .{g*>0}—=0. Condition, 
6.2,C is fulfilled and the theorem is proved, 





+c 20a) ap Ce 


6.14. Theorem 6.6, Let X be a Markov process subject to 
the requirements of theorem 6,5 in the space (F,p, 9). let 
F be a system of subsets of space (E,p, g) such that the N() 
condition is satisfied for each TE ¥. 





Then a Markov process X can be constructed which is equi- 
valent to X, is continuous up to departure from and has 
the following properties: for every o€ & there is either a 
TE€F such that x;(@)Er for all 4€(0, C@)) y or else for every 
TE there is a /, tE(F. w) such that Xt, @ér. 
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If the process X is ¥-bownded, X can be chosen so that 
F)=S 


Proof. Let [,, be a sequence of sets satisfying conditions 
3,12,4-3,12.C, We say that 9€.¢ if > is continuous until 
departure from ¥ and either x,€T,;for some m and all ¢€ 0,4), 
or there exists the sequence /,, ti(¥,¢) such that o(t,)ETn. 


We construct for each set [,, a function gq, as described in 
the proof of theorem 6.5, We shall write 9° for the result 
of substituting i and x: for x, in 9m. We put g= Dm 7 


o 
ye The function g is clearly measurable with ‘respect 
1 


to }. xX...) Xp --- and satisfies condition 6.1.4. When prov- 
ing theorem 6,5 we showed that P,,2{%,>0}=0 . Hence P, .{q° 
>0}=0 and condition 6,1.C is satisfied. It remains to 
prove condition 6,1.B, 


Let A=(t,, ..., ty...) S[s, 00), KE (s, 00), We put 
Em = 8h Pm) $= 8A (F) = lim Ene 


where #() is given by (6.23). We assume that 9@; 4.x) ..., 
ty Xn +.) == 0 SO that ImQi ty xp... te Xp...) = Ofor any m, 

We consider a function ? defined in the set A by the equa- 
tion 9()=x;. This function is uniformly continuous in 
Af{s. K] for every KE[s. §) and can therefore be uniquely exten- 
ded to a continuous function in the interval [s,§) . We put 
e(=9(s)for t€[0,s) and define ¢ arbitrarily in the interval 
t,2) 5 except for the proviso that 9(4)—.x; for all i, Let 
n= 2m 9) = '(F. 9). We show in the same way as when 
proving theorem 6.5 that, if & <A, then be ?® exists and 


belongs to Paw. It follows from condition 3" Ho 6G that Ting: 
GE\Iq. Therefore #,<in4,. Hence either &,-—. for a cer- 
tain m, or ticih<... <in<... <@. It may easily be seen 
that in both cases the function? that we have constructed 
belongs to #, This completes the proof of the first asser- 
tion of the theorem, 


It can further be seen that, if » is any ¥-bounded func- 
tion in @r and if x;= (4), the function constructed by us is 
also #-bounded, The second assertion of our theorem follows 
from this in accordance with the remark on theorem 6,1, 


6.15. Corollary. Let (£,p) be a s-compact complete metric 
space, # the s-algebra generated by all open sets, and 
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p(s, x, t.T)a transition function in the space (E.s, #). 


Let & be a normal system of subsets of space (E.s, #) 
satisfying condition 6,3.A, and let the N(T) condition hold 
for every TE, Then there exists a continuous ¥-bounded 
Markov process with the transition function P(s, x; 4,T). 


If condition 5.18.A holds, or if the transition function 
is stationary and satisfies requirement 5.19.4, the process 
is strictly Markov. 


The proof of this corollary follows exactly the same lines 
as that of remark 2 on theorem 6,3, 


Remark 1. The corollary obviously still holds in the case 
when the N(E) condition is satisfied. 


Remark 2, If the transition function P(s, x; 4,T) is complete, 
we can evidently choose a non-cut-off process as the process 
described in the corollary, Similarly, a stationary process 
can be chosen if the transition function is stationary. 


6, A Continuity Theorem for Strictly Markov Processes 


6.16. Theorem 6,7. Let ¥*) be a system of open measurable 
sets G,CG,c ... SG,¢... in the metric measurable space(E, 5, #) 
with #X #-measurable function p(x, y), and let xy—=(x, 4, 
oM,P,.) be a strictly Markov process in the space which is 
& -bounded and continuous from the right**) and satisfies 
conditions M(G,)(n=1,2,...). Letu<u<...<%<..-be an 
arbitrary sequence of random variables independent of the 
future and s-past. We put 


t(w)= lim +, (w)(w € Q) | 


Bio ee! j (6.55) 


*)The theorem obvicusly still holds if #¥ is any set system 


equivalent to the sequence {Gy}. 

**)If process X were not continuous from the right it could 
be replaced by an equivalent process that was continuous 
from the right, in accordance with remark 1 on theorem 6.3, 
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X2y (wa) > xz (w) (a.c e Q.. P,, x) (6.36) 


Proof, Let :(G;,) be the instant of first departure of the 
trajectory from G, (see sec, 3.8). Since process X is ¥- 
bounded, we have {7 <5(G,)} t+ 2., and consequently 


[E<ECGp) eq A.) t [er ae, He). (6.37) 
Furthermore, 
fe <E (Gy): 7% *} =U (6.38) 
where 
C= AU fem: x)>4, << tGy}. 


We choose an arbitrary 4>0 and put 
F¢, «) = etme (4) Xe, (Xtou) 0 Sp Xe4u) 
(tE [s, co), o € Q,). 
Obviously, 


cheU N) {row >A}. (6.39) 


M=1 N= 


Since X is continuous from the right, whilst the sets G, are 
open, we have 


f(é, 0) = on Xe, ) La, t+) Pp Xu)» 
where A is the set of all rational numbers of the interval 
{0,z). It is clear from this that the function f(t, ) is 


measurable with respect to the so-algebra x, and by theorem 
5.2: 


Py, off (= 0) > 2 } = Mg, oF (ty %y)> (6.40) 
where 


F(t =Puy{ fu 0) >=}. 


By lemma 6.3, we have for any yC€G,;: 
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i 


F(a 9) < 28). (6.41) 


We obtain by combining (6.39), (6. 40) and (6.41): 
P,...(Ch) <a (4). 


1 
In view of the M(G;,) conditions, ag, (0 as 4-40, so that 
Paso . On taking into accdint (6.37) and (6, 38), we 
(6 36 


Remark, Theorem 6.5 holds for any strictly Markov process 
which is continuous from the right and satisfies the M(E) 
condition, For we can put in this case G,=...=G,=... 
=E, 


6.17. We shall in future describe the Markov process X= 
(x,.%, Mt, Ps,2)@S quasi-continuous from the left if, whatever 
the random variables %, t:, ..-, tT, ...independent of the future 
and s-past, and whatever the subset © of set ©, the rela- 
tionship 


Ta(w) tro) <f(w) (ww EB), (6.42) 
leads (for every x€£) to the relationship 


x, +x, (8.0.9, P 


2 5, x). 


We arrive at the following proposition by using theorems 
6.7 and 5.9 in conjunction with remark 2 on theorem 6.3 
(sec, 6.9): 


Theorem 6,8, Let (E, p) be a s-compact complete metric space 
and @ the s-algebra of all Borel sets of the space (E, p). 
Let  ={G,} be a sequence of open sets such that the closure 
of G, is contained in Ga,, and G,+£. The transition func- 
tion P(s, x; ¢,T) in the space (E, p, #) will be assumed to 
satisfy conditions 5.18.4, 6.3.A and M(G,)(1=1, 2,...). 





Then a process can be constructed with the transition func- 
tion P(s, x; #,T) such that it is strictly Markov, continuous 
from the right, quasi-continuous from the left, devoid of 
discontinuities of the second kind and ¥- bounded, 


Remark, If the transition function P(s, x; t, T)is stationary, 
conditions 5,18.A can be replaced by condition 5,19.A, i.e. 
the requirement that the transition function be of Feller's 
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type. 
7. Examples 


6.18. The phase space in all the examples below is a s-con- 
pact complete metric space, whilst the cs~algebra # is gene- 
rated by the system of all open sets of this space, We can 
therefore base our discussion on the corollary of theorem 
6.6, 


6,18.1. We have for the transition function P(s, x; #4, N= 
Sp [x + u(t—s)](see sec. 4.2.1): 


at (6) = 2%,(8) = 0, if a<t. 


In accordance with the remarks at the end of sec. 6.15, there 
is a stationary non-cut-off continuous process on a straight 
line corresponding to this function. We shall describe this 
process as a determinate motion with velocity v. We can 
easily show that, for the process, 


P,{x,=x+2 for all ¢>0}=1. 


6.18.2, We have for the Wiener transition function con- 
structed in sec, 4.2.3: 


oy a 2 
Px tthe CG ay? fe Maz, 
lz[>«e 
where 2? denotes the scalar square, and |z| the length of 
vector 2, We get from this, after some simple calculation 


Pix; th. TO) =euF (FZ): (6.43) 


where c, is a constant, and 


o 


r? 
F(r)= f e 2 r®-"dr, 


r 


It follows from (6,43) that 
e (e) — = 
fol O=aF (Fe). 
By L'Hospital's rule, F(r)—;n-2,-F as r—+co, so that, as 
610: 
n-2 id 


at (8) ~~ eys"-2 8 Fe 2, 


It is clear from this expression that the N(E) condition is 
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fulfilled, In accordance with the remarks at the end of sec, 
6.15, we can conclude from this that there is a stationary 
non-cut-off continuous Markov process which corresponds to 
the Wiener transition function. We describe it as a Wiener 
process, It may easily be seen to be of Feller's type. It 
is therefore strictly Markov (see theorem 5,10), 


6.18.3. We shall now investigate the transition function 
described in sec. 4.2.4 in the interval [0, 0). We notice 
that, for any a>0,x«>0,¢>0,A>0: 

Pit. x; t-+-h, (a, co)) = 
1 ¥ 1 Ox 
Sy oe. . é WS TE VE (6.44) 
lyVi-al<az 

It follows easily from (6.44) that the system ¥ of all 
sets [a, cc) (a2>0) satisfies condition 6,3.A.( [a22° ) can be 
taken as E,), System # is obviously normal,'”Simple calcu- 
lation similar to that of 6.18.2, shows that, for «<a, 

1 e 
a, co) (3) ~ ce“13% oF, 


TheN (T)condition thus holds for any [T€#. and by the corol- 
lary of theorem 6.6 (see also remark 2) an #-bounded contin- 
uous stationary Markov process exists with transition proba- 
bilities 4.2.4. This process is clearly of Feller's type and 
is therefore strictly Markov, It can be shown that, for any 
s>0, x€E: 


lim x,(w)=0 (a.¢, &,, Py 2) 
tat (w) 


We may describe the process as a one-dimensional Wiener 
process with a break at the point 0*). 


6.18.4. For the transition function of sec. 4.2.5 we have 
ar(a) cenlgt gE » and some continuous Markov process there- 
fore corresponds to the function, In accordance with remark 
2 (sec, 6,15) » We can choose a non-cut-off stationary process 
for the process. We shall refer to this as a one-dimensional 


Wiener process with reflection at the point 0. 


6.18.5. We shall now construct an example of a continuous 


*)What we have described is not in fact one process but a 


whole class of processes, For the sake of clarity we can 
choose from among these the canonical process which is subor- 
dinate to all the rest, 
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Markov process which is not strictly Markov, 
We take a stationary transition function on the real line 


E=(—~x, +029)with the usual metric p(x, y)=|y—x], the func- 
tion being defined by 


vEvea { p[-ygen] @ if 2x0. 


ty &), if x=0. 


P(s, x; t, T)== 


This function is complete and satisfies the N(E) condition, 
It therefore corresponds to some non-cut-off continuous 
stationary Markov process X=(x,, of, P,, 2). 


The instant <—£(G) of first departure of the trajectory 
from the set G=F\{0}is a random variable independent of 
the future and 0-past, For, if A is a denumerable set 
everywhere dense in the segment [0, ¢] , we have 


(>1=U Pflxl>ah com 


We put w€A if there exists a ¢ such that x,(w)—0 for all 
vet, It may be observed that, if x+0, Py .(A)=0. This 
follows from the inclusion A <U! te, = 0} and the equation 


Po, 2 ixXn,=O}=—=P(0, x; 2, x)=0."™ 
On the other hand, Pp, .(4)—=1. For 


AD>f){x,=0}, 


rEA 


where A is the set of all rational non-negative numbers, 
Since for every rs 


Po, 9 {%- = 0} =P(0, 0; 7, = 1, 


we have 
Poof f} =O} =1 
TEA 
and all the mores 


Po, (4) = 1 
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It may easily be seen, moreover, that ',A—Afor any u, so 
that #,A=A. 


If the process X were strictly Markov, we should be able 
to write, in accordance with (5.40), 


Po, « {82-4} = Mo, 2Po, », (A). (6.45) 


But x,=0, so that the right-hand side of (6.45) is equal 
to unity for any x*), At the same time, the left-hand side 


is equal to zero for x#0. 


*)It is easily proved that Po 2{%:}=1for all x, 


ADDENDUM 


A THEOREM REGARDING THE PROLONGATION OF 
CAPACITIES AND THE PROPERTIES OF 
MEASURABILITY OF THE INSTANTS 
OF FIRST DEPARTURE 














1, A Theorem Regarding the Extension of 
Capacities 


1. A compact Hausdorff space with a denumerable base*) is 
referred to as a compact. 


Locally compact Hausdorff spaces with denumerable bases 
will be described as semi-compacts. Some properties of such 
spaces will be listed, 


1.4, A semi-compact is a c-compact space, 


1.3, <A topological space is a semi~compact when and only 
when it can be got by discarding a point from a compact, 


1.C, A complete metric can be introduced into any semi- 
compact, 


1,D. Each closed and each open set in a semi~compact can 
be written as the sum of a denumerable number of compact sub- 
sets, 


Proof of 1A. Let CS € be a denumerable base of the semi- 
compact (F, @). We write C's for the subsystem of system @ 
consisting of all the elements of @ with compact closures. 
Let x€E&. In view of the local compactness of (£, @), there 
exists a neighbourhood U of the point x with a compact 
closure, Since @ is a base, there existsG€ @ such that 


*)A subsystem @ of a system Cis said to be a base of the 


topological space (£, @) if every set Te @ can be written as 
the sum of elements of @, 
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x€Gau. Clearly, G€ @’. Thus every point x€E belongs 
to some set GE @’, and @’ is a covering of space F, The 
closures of the elements of @’ form a denumerable system of 
compacts adding up to the sum £, 


Proof of 1.B, Discarding a point from a compact evidently 
leads to a semi~compact, We show that a compact can be got 
from any semi-compact of (F, @)by adding a single point a, 

We put £,— EU {aland write C, for the system of all subsets 
appearing in © and of all sets of the form {a}U{E\T}, where 
Tr is any compact subset of space (F, @), It may easily be 
seen that (£,, @:) is a compact. 


Proof of 1,.C. In view of 1.B we can assume that F=E,\ {a}, 
where (Z;, @;) is a compact, into which some metric p(x, y) is 
introduced*), This metric may easily be seen to be complete. 
The expression 


p(x, =e y)+le(s a) '—p(y, a)7"] (4, VED) 
is readily seen to define a complete metric in EF, 


[-*) 
Proof of 1,D, In accordance with 1.4, F=[J &,,, where 


mal 


the £,, are compact sets, If TL is closed, we have 


r=UTlEn 


mal 


and the TE,, are clearly compact, Now let G be open, and 
let p(x, y) be any metric in (£, @) (existing by virtue of 1.¢). 
We put 


Th={y:0(x, E\a)>>}. 


Obviously, IT,, is closed and 


The sets T,E£, are compact, 


2. We shall write #(E, @)for the class of all compact 
subsets of the topological space (£, @), #.(E. @) for the class 


*) The proof of the metrizability of any compact can be found 


e.g. in Ref, 25, 
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of all subsets which are the sums of a denumerable number of 
compact subsets, and .#.s3(E. C)for the class of all subsets 
expressible as the intersections of a denumerable number of 
sets of #.(E. C). 


We shall describe a subset I’ of the space (EF, @) as analy- 
tic if, in some compact (F’, @’) a set BE co%4(E’, @’) can be 
constructed which permits a continuous mapping on to Tr, 


Lemma 1. Every Borel set in the semi-compact (£. @) is an 
analytic set, 


Proof, Let & denote the class of all analytic sets in 
the space (£, @). 


1°, We show that, if T,€C/ (n=1, 2, .-.) and 


= U QT, 


n=l 


then TES. 


By hypothesis, there exists for any a” a compact (E,, @,) and 
a set B,€o%.2(E,, @,) admitting a continuous mapping f, in 


The 


Let 
‘me co 
E= UE, 
n=t 
and let @ be the.glass of all subsets of the set E express- 
ible in the form (JU, , where Un€@,. The topological space 
(E, @) is a semi-dompact, so that (see 1.B) it can be got by 
discarding a point from some compact (E’, @’). We put 
oO 
B= B, 
n=1 


and define a mapping f:B-+E by the expression f(d)=fnlx) » 
if x€B,. It is easily verified that BE c#a(E’, @’) and f is 
a continuous mapping of B on to I’. 


2°. We now show that, if 
TreéESP (n=1,2,...) 


and 9 


r=f} rT, 


nei 
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then TES, 


Let (En, Gn) B, and f, have the same meanings as in sec, 1°, 
The space 
(E, C')= 
=(E, X E,K «6. K Eg XK oes C1 XK CoX% 0 K CaX +03 


is a compact. The set 


B=B,XB,X...XB,X... 


is the intersection of sets E,x%... XE, 1B, XE Kee. 
(E’.@’)-Consequently BE o%3(E’, @’). Pere ee Co¥as 


We take the mappings f,:B—+E defined by 
Fn (Kir eves Xpe os -)=fa(%n)- 


Bach of these mappings is continuous, and the set 


BSlx:AWQ=hO=... =f) = -..)} 


is therefore closed with respect to B, i.e. B=BB , where 
B denotes the closure of B in £’. The set B is compact, 
Therefore B=BBE o%,(E’, @’). The mappings f, all induce 
the same mapping f of set B in E, Obviously /f(B)cT,-for any 
n, So that f(B)ST. We show that f maps Bon to I’. For 
let c€l., There exists x,€B, such that f,(x,)=c for any nr. 
The pointx (x, x2, ..., X,,...)of E’ belongs to B and f(x)=c. 


3°, &11 the open and all the closed subsets of (E, @) 
belong to #, 


For & contains any compact set I' (since we can put E’=B 
=[ and take the identity mapping for f), By 1.D, every 
closed and every open set can be expressed as the sum of a 
sequence of compact sets, Our assertion therefore follows 
from sec, 1°, 


4°, We pot rey’ if TES and TEX. It follows from sec. 
1° and 2° that § is closed with respect to the operations of 
denumerable summation and denumerable intersection. Since it 
follows obviously from TE that TE S’f’is a s-algebra, In 
accordance with 3°, /’2@. Consequently /’35(@), i.e. it 
contains all the Borel sets of the space (£, @). 


3. Let F be any system of subsets of the semi-compact 
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(E. @) such that it contains all open and all compact sets, 
We shall call the function «, defined on ¥ and taking values 
from the segment [0, 1], a capacity if the following condi- 
tions are fulfilled: 

3.4, IfT, Sf, and 7, ECF, then o(l,)<¢(f). 

3.B. Iff, fT and T,€F, thenTe and o(f,) t eM). 


3.0, Given any compact set [ and any e«>0, a GE@ can 
be found such that Gar and 


¢(G)—¢(P) <e. 
3.D. iff, 7, 3€¥ and Per, then 
e(@UB)—eTUB)<et)—¢l. 
The following property is a consequence of 3,Ds 


3.E, For any Por(i=—1,2,..., n)of F: 
n n ne n aS, 
o( n) —¢% (U fi) <Rle@)—e Od). 
1 1 


n nm n 
a( n)— ¢ (U i= Ste@iusy— eu Ro. 
1 1 


For, 


where 
Wet KUT U eae E eS i1 8s eascay, 


For each TC Ewe shall write ¢*([) for the lower bound of 
?(G) with respect to all open sets G containing I, and .(T) 
for the upper bound of all 9 (Pf) over all the closed sete fu 
contained in IT, 


Our purpose is to prove the following theorem regarding the 
extension of capacities. 


Theorem 1, Whatever the capacity ¢ in the semi-compact 


(E. @) » we have for any analytic (and in particular, any 
Borel) set I': 


1) = 9"(1). 
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4. We shall need some lemmas before proving theorem 1, 


Lemma 2, For any sets frercE(t=1, 2,..., n) 
n no n 
¢ (U n)— e (U fi) <tr @)—9 Gi. 
1 


If T, tl, we have 
e(P,) t oO). 


Proof, Let e«>0. We choose open sets G, (i=1, 2,..., 
nn 


and @ such that T;S6;, GaUf, ana 
1 
e(G)<eT)+e @=1, 2,..., 9), | 
n 
o@)<e (UT) +. 
1 


= ~ ~~ 


We put @,;=GG,» Obviously G3G,;>F, and 


We have on the basis of cos: (4) and 3,E: 


H(d)--") <o(G2)-(09)< 


<Dle@—-eG@l+e< Die G)—e EN+G4 Ve 


Since « is arbitrary, (1) follows from (5). 
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(2) 


(2) 


(4) 


(5) 


Now let r, +L. For any e>0 we can choose open sets G, 


such that G, >YF,, and 
2 (Gn) < oe t+" Pn). 


n 
We put Go — U G,. 
1 


In view of (1): . 7 
9 (6) —¢C,)=¢ (U G,\—¢" (U n) = 
1 
< oy 


(6) 
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We put 
e=Ya=Uc™. 
1 1 
By 3.B: 
¢ (a) t 9(0). (7) 
From (6) and (7): 
9G) — lim 9° (En) <e. (8) 


Since ¢(G)><*(7,), (2) follows from (8). 
Lemma 3. If TE€#a(E. @), we have 


(LP) = 9° (I). 
Proof. Let TE #a(E, C). Then I can be expressed as 


T=f)t, (.€0%.(E. ©). 


n=1 
We can choose the sets T,€o%#°(E, @) such that ré+7,. We 
assign any positive e« and put 


Be = r'r. 


Obviously, Bf ¢ Tr and by (2), »*(BY) + e*(T). We select &, so 
that 


o()— ¢*(Br) < 5. 
We put 
Bg = BETS. 


We have BY t BE. By (2), o*(13) t o*(RH) . We choose & so 
that 


or (Bt) — 9 (0H) <. 


On continuing these constructions, we get a sequence of sets 


rr 
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such that 
(Bit) —# IR) <S. 
Thus for any 73 
oO) — 9 (By) <e+ ear Gs (9) 
We put 
U3 


B 27) Bin rr T” 
e n tee ehaiiels 


n=l e 
A,= Tere... rie, 
k oO 
Since r"GVT, and f)I,=I, we have 
1 
a y's cr 
n = 
1 
and 
=r 
1 
is a compact set. By 3.C, we can choose an open set Gd B, 
such that 
¢(G)— 9(B) <e. (10) 


We observe that 
Beh = AT and A, | B,. 


Hence (in view of the compactness of A,), a8 from a certain 
nyB"c A, SG and by (9) and (10): 
k, 
(0) <9 (BM +e<e(G)te<9(B) +2. 
Since B, is compact and contained in I, we have 


2 (BR) <9, (L) and 9*(L) < o, (T)-+ 2e. 


It follows from this, in view of the fact that « is arbitr- 
ary, that ¢*(I)=9,(). 


5. Proof of theorem 1, Let IT, be an arbitrary analytic 
set of the space (£, @). We can construct in some compact 
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(E, @')a setBe #., (EG )admitting a continuous mapping f on 
the set I, . We take the space (£, @) =(EXE. ex @’) (it 
is clearly a semi-compact ) and write p for the projection of 
E to E, i.e. the mapping given by 


p(x, x)= x (x€E, x’ EE’). 


The set of all points F(x’). x’) (x’€B) will be denoted by fF, , 
If pBEF, we put le F and 


9(f) =9(0). (11) 


1°, We show that the function of) FEF) is a capacity in 
the space (£, @). Firstly, contains all the compact and 
all the open sets of the space &, @) . For Pa) is open if 
I is open, and compact if P is compact, Moreover, the 
mapping p preserves the operation of summation and the inclu- 
sion relationship, Hence, since conditions 3.A4-3.B and 3,.D 
hold for the function ¢, it follows that they also hold for 
the function o, We prove 3.C by observing that, if [isa 
compact subset in (Ff, @) and if GE @ is such that 


2(G)—elp@l<e, 
the set G=G XE’ belongs to e , and at the same time we have 
g@—e®<e. 


2°, We show that Tié.v5(— @). To start with, it may 
easily be deduced from the continuity of f that 


R=(EX BT, (12) 


where To is the closure of T, in (& @). In accordance 
with 1.D, the closed set T) in the semi-compact (E, @) can 
be written as 


where K,€o#°(E. @). On the other hand, 


E=UEm B=f) (si. 


kml tet 


where sets E,, and By are compact, We have from (12): 


h=N U (mXBi)K,. 


kel im, net 
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The sets (Em X Bi) K, are evidently compact, so that Ft wx (E, 
C). 


3°, It follows from 2° and lemma 3 that 


9, (Fo) = 9° (h). (13) 
We observe that 
eTo= sup 9(T)> sup _ efpPi=9,t) 
Tes (E, @) Tex (Ez, @) (14) 


Tet, Tot, 


e*(Ty)= inf 9(G)< inf 2[p(Gyj= 9"). 
Gee Ge 
G2, G2 tT) 
It follows from (13), (14) and (15) that o%(f))>¢*(f,), But 
since 9 (Io) < 9*([,) in view of 3.4, we have 9,(To)=9"(Io) « 


2, Measurability Theorems for the Instants 
of First Departure 


6. Let X¥=(x,, %, ofj, Ps, x) be a Markov process in the measur- 
able space (£, #). For every w€2, we write Fi=Fi(w) for 
the segment of the trajectory during the time [s, t], i.e. the 
set of points *,(), where 4€ls, t], 


If the process xX is given in a topological measurable 
space, we shall write Fi for the closure of the set Fi}, 


lemma 4, Let X be a Markov process continuous from the 
right in the topological measurable space (E, @, #). Then 
for any closed set T€ @ and any U<cscti 


{Ficl) = {Fist} esi. (16) 


Let UC @#. We assume that one of the following condi- 
tions is fulfilled: 


6.4, (E, @, #) satisfies condition 1.9.B and the closure of 
U is compact. 


6.B. (£, C. #) is metrizable,#2C and ENU is compact, 
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Then we can construct a sequence of measurable closed sets 
T, t U such that, for anyO<s<ts 


{Fisty| t {Fieu} (17) 


and 
{Fis U} EN}. (18) 


Proof, Let I be a measurable closed set and let A be any 
denumerable everywhere dense subset of the segment Is, ¢] , 
containing the point ¢. Obviously, 


{FicT) = i= => {x,€T}, 


r€a 
so that (16) holds, 


Furthermore, let U be an open measurable set. In case 
6.4 we take the function f(x) defined by condition 1.9.B, In 
case 6.B we put f(x)=p(x, J). The sets 


1 
={x:f@>5} 
are closed and measurable and Tf, +Uu. Hvidently, 


ao 


Hou} aUlMenl. 


We can prove (17) simply by proving that the converse 
inclusion also holds, All we have to do for this is to show 
that 

{Fis U} & { inf f(x) > 0}. 
2eF} (19) 


In case 6,A, this inclusion follows from the fact that the 
continuous function f(x) attains a minimum on the compact set 
Fi. In case 6.B, the continuous function p(y, Fi) attains a 
minimm on the compact set U. If 


PU =, 
this minimum is positive and we have 
inf f(x)=p(FZ, T)>0. 
zere 
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As already mentioned, (17) is a consequence of inclusion 
(19) just proved, Expression (18) is a fairly obvious conse- 
quence of (16) and (17). 


7. We put A€oV; if, for every measure sp on the s-algebra # 
there exist sets A, and 4, of Wisuch that 4,6 AG A, and 
Po,y (Ar) = Po, u (Ad)*) « 

Theorem 2, let (EF, @) be a semi-compact, let # be the c- 


algebra in E containing @, and let X be a Markov process 
continuous from the right in the topological measurable space 


(EF, C, #). 
Then for any Borel set [ and any 0<¢s<t: 
(FoT}e dt. (20) 


Given any Borel set I, any measure p on the s-algebra # 
and any 0<s<t, we can construct a sequence of closed sets 


Lelhs...¢hec...er 


a= 


and a sequence of open sets U,3U,2... 2DU,3... BF with 

compact complements such that 
P,{PiSt,} t Ps. {Plcrt, 
Pen {Aisu,} + P, (Piss t}. 


Proof, We fix the 0<s<i and putTC¥ if 
{Pi ENT] CoV. 


In view of lemma 4, the system ¥ contains all compact and 
all open sets. Having fixed a probability measure » on the 
c-algebra #, we put for each TES: 


9(T)=Py,. {Fel #9} =P, .{Q\[Ae ENT) — (22) 


We show that the function 9 is a capacity, i.e. satisfies 
conditions 3.A-3.D. It is obvious that 3,4-3.B are fulfilled, 


*)Cf, foot-note on p.55, 


**) This expression is meaningful because the set in brackets 
belongs to-N;G N° and, in accordance with sec, 2,3, the 
measures Ps. can be extended to dl, 
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Conditions 3,C follows from relationship (17) which we proved 
in lemma 4, It remains to consider condition 3.D. 


~ 


We notice that, iff, FP, pég#and Per, then 
sTubB= 
=P,,. (Aiur) +0} =P, (Fir ou(Fineo}—= — (23) 
=P,,, (Ait + o}+P,,. {FB + o]—p,. (PP 40, ABs o}— 
=e) +2()—P,, {Pit 49, PIB oh. 
Similarly, 


eFub)=2@)+9(8)—P,,, (Pi +0, Pip zo}. (24) 
We get from (23) and (24): 


—eTUB)+¢PuR+eM—¢H= 
=P, {AT=0, AP +o, PIB + 9} >0, 


whence 3.D follows, 


Let T be any Borel set, Then I is also a Borel set and, 
by theorem 1, we can find for each m a compact set I” and an 
open set U" such that 


Perey 


0<9U)—eO<s. 


It follows from this last expression and (22) that oarai* 
and. 


The sets 


satisfy the inclusions 


TjS...S1,¢6...6TS...6U,6...6U,, 
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whilst 
0<P.. {SU} —P, fer} <t, (25) 
We put 
4,=U (Fis T,}. Ap =f} {Ac Un}. 
n=l nal 
Obviously , 


AS {Aier} SA, (26) 


In view of lemma 4, A, and 4, belong to NY. Farthermore, 


P A,) = li pt 
aoe ( ») ee Pa, # (Fi s Tr}. (27) 
U, 


Pa,» (Ar) rns Psa {Fi S a} 
and by (25): 
Pe, » (4) = Py, » (A). 
Relationship (20) therefore holds, 


We conclude from (26) and (27) that relationship (21) also 
holds, 


Remark, The reader will observe on going through the 
proof that the theorem still holds in the case when I is 
the complement of any analytic set. 


8. Let X=(x, %, off, P,.,) be a Markov process in the measur- 
able space (£, #). The instant of first departure after s 
from the set I *) is given by 


Ram { wl rer} if S@)>-s, 
S, if f)<s (28) 


(this may easily be seen to be equivalent to the definition 
given in sec. 3.8). 


*)This magnitude can also be called the instant of first 


reaching the set E—I. 
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If process XY is given in a topological measurable space, 
the function 


sm _f{ sup{t: Fist} if %@)>s, 
LO =| $, if Clo)<s (29) 


will be described as the instant of first departure after 5 
from inside the set I *). 


Theorem 3, Let X be a Markov process continuous from the 
right in the topological measurable space (E. CC, #), 


If T is a closed measurable set, we have for any 0<¢s<t: 
83) >t} = (CL) >t] EV iso. (30) 


If TE @# and if one of conditions 6,A-6,B is fulfilled, 
we have for any 0<¢s<t 3 


(EP) > thE Wi. (31) 


Finally, if (£. @) is a semi-compact and #2 @, then for 
any Borel**) set [f and any 0<¢s<¢ 


*)Alternatively, we can call ¢,(f) the instant of first con- 
tact with set £-r. 

We defined in sec, 3.13 the instant of first departure 
after s from inside the open set r as the instant of first 
departure after s from the normal system + for which 


U T=G. 
rer 


If G has a compact closure, the earlier and the new defini- 
tion are easily shown to lead to the same function (ef, the 
proof of lemma 3,8), 


**)And in fact for any set 1 whose complement is an 
analytic set, 
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(8. (2) > £}€ iso *)- (32) 


In this case, we can construct for any s> 0 and any measure 
pon # a sequence of closed setsl,...S1T,¢... contained 
in T and a sequence of open setsU,2... 2DU,3... contained 
in T with compact complements such that 


Psu (Fe (Tn) t §(M)}) =P. 18, (U,) | SD} = 1. (33) 


Proof, Let &—=#(P), §&—§ (1). We have for any u>z and 
any integer m: 


A A “2 ( As 
{E> 4, eat iat. 2a (34) 
Suppose that [ is closed, Then by lemma 4: 


une sTledr, 1- 
ty toy 
Consequently, 

{53 >t, QE fh 


for any v>z, so that {%>t}EWNiio , Since t,—1, fora 
closed set, relationship (30) holds. 


Now let I be open and one of conditions 6.4-6,B be satis- 
fied. It follows easily from relationship (19) that we 
deduced when proving lemma 4 that 


Since by lemma 4: 


(FIST}E Ni, 


*)We put acWNi.o If ASY, and goed}, for any v>t . 
Obviously, ,j}., is a s-algebra in the space »,, The c- 
algebra ,\"/,9 1s similarly defined, It will be noticed that 
AcdWi,o when and only when there exist for every measure » in 
the c-algebra # setsa’, AveWi.vsuch thats’ SAG avandP,,, (4’)= 
P,,.(4”) (ef, foot-note on p.55). 
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relationship (31) holds, 


Finally, let (F, @) be a semi-compact, let #> Gand let Tf 
be an arbitrary Borel set. By theorem 2: 


p38 int 
{Pee Sr peels. 
We can therefore conclude from (34) that 
{>t} Ed i+0. 


We now number all the rational numbers 1r,, ry, .... ln +++. By 
theorem 2, for each k we can choose sequences 


MWe...eMc...ere...cutc...cut 
(the Tz being closed, and the U% open with compact comple- 


ments) such that, as n—-+oo, 


ae he ae t Pan ite eT}, 


r= 


P,,. (Fr, & Un} + P,,, (Fr, ST}. 


We put 
n n 
Tn=UTs. Uz=f) Un 
kel k=l 
Since 
re er,¢..., U,B...BU,R 
and 


Renercu,cut, 


for n>k, we have for each & as n->co: 


Psp (Pr, SP a} t Poy (Fe, S 7), (35) 
Psa (Pt, SUp} | Poy (Pt, ST}. (36) 


Clearly, 


BOy<...<&Q<.--<&M<--- : 
wee SE(Un) < --- Sbe (U2), 
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so that 
{& x) # & (1) =U NECI<n<MS 
pe (37) 
Ss U Nils TIN (4, =f,]}- 


It follows from (35) and (37) that 


Ps, (5 (T,,) t & (T)} = 
Similarly, it can be seen on the basis of (36) that 
Parl&Un) + &@) = 


9. The functions §&(I) and §, (T) do not increase as s de- 
creases, with the result that there exist the limits: 


Se+0(P) = lim & (DP), 
tys 

$40 (I) = lim &(D). 
tye 


Theorem 3°, let X be a process continuous from the right 
in the topological measurable space (E, @, #). 


If [ is a closed measurable set, we have for any Ogs<ct? 
(Bo (L) >t} = {Eero ) >t} EW fro. (38) 


If T€@# and if condition 6.4 or 6.B is fulfilled, we 
have for any 0<¢s<?t: 


(8.400) >t} Eo. (39) 


Finally, if (£, @) is a semi-compact and #2@ , then 
Gero) > tHE Mis0, (40) 


for any Borel*) set [ and any 0<s<?t. 


In this case, given any s>0 and any measure » in @, we 


*)And for any set [ which is the complement of an analytic 


set, 
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can construct a seauence of open sets U,2>...3U,3...31T 


such that their complements U,,..., U,,... are compact and 
Pasa euo(Ua) § fe49)) = 1: (41) 


if r= 0"), we can construct a sequence of closed sets I, 
e...6V,¢...¢e7 such that 


Psy Es40Cn) t Esso} =i. (42) 


Proof. Let [ be a closed measurable set. If s<uct, 
we have by (30): 


ED) >t =) > He NtioS Miso. 


We conclude on letting 44s that (38) is satisfied for 
s<t. If we now let ¢ tend tos, it will be seen that (38) 
also holds for t=s. Expressions (39) and (40) may be veri- 
fied similarly. 


Relationship (42) follows from (33) on taking into account 
the fact that 2,.,(f)>(f) for any set T and Ps, {5 310(L) =&, (T)} 
=1lif »m=0. 


We still have to prove (41). We choose a sequence s, | s 
and put 


ba(l) =Ps, . (xe, €T} (TE #). 
Using theorem 3, we construct for each 1 a sequence of open 


sets U,2U,2...2U'3...29, having compact complements 
and such that 


Pi bon fe (ui) 1 Fay (ry} = 
By (32): 


ae le, (Un) 4 ey (1) € ole, 


*)The following example shows that this condition is essen- 
tial, Let xX be a determinate motion on a straight line with 
velocity 1 and let T=(0,0). Then for any closed set TSr: 


Ps, 0 <En+0 a =0, beo(I') Sobel. 
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By theorem 2.1%: 


Pie (A) = My, 2Ps,,, ey AG 


so that 
Py, 2 (4) = Mg, oP... Zs, (4) = P.., *n (Api. 


Thus we have 
&,(Ua) 4 i, (P) (ase. & PL. 


a 


for each n aS k-+oo. We put U;=Uin...nuj. Clearly, 
U,3...BU,2...3T and 55, (Ux) < Be (Ui) for k>n. Hence 


5a, (Ux) tL 5s, (T) (a,c. 2; Psa (43) 
for any 1 as k->so. We have, moreover, for any x and k, 


ta, (Oi) > Fo (Ux) > Ssa0 (Ty 
and by virtue of (43): 


Say The in 5s 20(Ux) > Ferg (P) (a,c. 2. P 


sy): 
Since 
Beve (t) = 5e40 (T), 


we get 


yim Ss10(Un) = Ss40(P) — (a.¢. 2, Py). 


10. Theorem 4, Let (£. @) be a semi-compact, # a s-algebra 
in £, containing @, and X a Markov process continuous 
from the right and quasi-continuous from the left*) in the 
space (E, @, #) such that 


Miso S off} 
for all 0<s<i. Then 


P,, x (,(L) * §,(0)} = 0. (44) 


*)see sec. 6.17. 
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for any Borel*) set [T and any s>0, x€—. 


Proof, Let YU be any open set. By theorem 3, a sequence 
of closed sets T,&...S7,&...GU can be constructed for 


any s>0, x€E, such that 
Peel Cots) 1: (45) 


In accordance with 1.C we can introduce a metric p(x, y) into 
the space (E, @). We put 


F.=T,Ufs ip, U)> zh. 


~ 


Obviously, T,¢...c¢P,¢...q@uand T,cf. Hence 


tee(P,) t &5(U)} S i f,(P,) t £,(U)} 
and 


Ps,2 {54(Pq) t $e(U)} = 1. (45) 
It is clear from theorem 3 that the {, (F,) are random variables 


independent of the future and s-past, and it follows from 
(45), since X is quasi-continuous from the left, that 


FEM FE (B02 82 icy Poa): (46) 

~ Pan ~ s(x. ~ Upc 

But :,(f,)=?,(r,)for the closed set I’, , and ” (4, wy Y<z 
since X is continuous from the right, It follows from this, 


in view of (46), that 


oe yp U)=0 (@.c. 22 uy Py, z)- (47) 
Since 


fF(U)=3,(U)} S [x2 ey EU} = (0 (%8, cay UY) > 9}. 
it follows from (47) that 
P,, et E4(U) #8, (U)} == 0. 


This proves the theorem for open sets. 


*)See footnote on p.1g9l. 
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Now let T be any Borel set. By theorem 3, for each s>0 
x€E, @ sequence of open sets U,3U,D...3BU,3...D can 
be constructed such that 


Pye f3e(Un) ¢ Fe) = 1. 
By what has been proved 
Poe {te(Uny=is(Un) for 2=1, 2, yl 


Hence 


Py, x {Es(Un) | 25 (T)} = 1. (day 


But 
5s(Un) > ba (LT) > £52), 


and (44) therefore follows from (48). 


Remark 1. let (E,@) be a semi-compact and P(s,x;t,r) any 
transition function in the measurable space (£,3(@)) that 
satisfies the conditions of theorem 6.8 (sec. 6.17). Then 
it follows from theorems 6.8 and 5,11 (sec, 5.21) that a 
Markov process X exists with the transition function P(s, x; 

t, T), for which all the conditions of theorem 4 are fulfilled, 
so that (43) holds. 


Remark 2, Let X be a process satisfying the conditions 
of theorem 4, It follows from theorems 3 and 4 that, if we 
take the c-algebras 8; in the sense indicated in the foot- 
note to p.55, all Borel sets are admissible (see sec. 3.8) 
for process X (and indeed all sets which are the complements 
of analytic sets). 


The same statement holds true without the requirement of 
quasi-continuity of the process from the left if the func- 
tions :,([) are considered instead of functions &(F). 


SUPPLEMENTARY NOTES 


Chapter 1 
The reduction of the basic ideas of probability theory to 


measure theory is due to A.N, Kolmogorov (Ref. 19). 


Chapter 1 gives a brief résumé of the results of measure 
theory which are used in the present book, A more detailed 
treatment can be found in the monographs by Halmos (Ref. 23) 
and Loeve (Ref. 20). 


Article 1, The concepts of a =-system and 4-system of sets 
and of an -system of functions are introduced for the first 
time. Lemma 1,5 is due to Doob (Ref. 7, supplement, theorem 
1.6). 


Article 2, The proofs of propositions 1.4,A-1.4.C can be 
found in the works of Halmos (Ref. 23) and Loeve (Ref. 20). 


Article 3. The general definitions of conditional proba- 
bility and conditional mathematical expectation were given 
by Kolmogorov (Ref, 19), We follow Doob (Ref. 7) and Loeve 
(Ref, 20) in the formulation of these problems, The only new 
feature here is that we consider functions and s-algebras 
defined in some subset of, instead of throughout, the space 
2, (Doob mentioned the possibility of carrying over the 
normal definitions to this case in Ref. 8). 


Article 4, The definitions relating to topological and 
metric spaces can be found, for instance, in works by Haus- 
dorff (Ref, 25), Loeve (Ref. 20) and Halmos (Ref. 23). The 
concept of topological measurable space appears to be investi- 
gated for the first time in the general form. 


Article 5, Theorem 1.1 is often called the theorem on the 
extension of measure, It is generally proved in a somewhat 
different form (see, e.g. Loeve, Ref, 20, 4.1 or Halmos, Ref. 
23, sec, 13), for the case when the system @ is an algebra, 
i.e, contains the complement of each set as well as the set 
itself and the intersection of every pair of contained sets, 
This case is readily obtained from the more general case 
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considered in theorem 1.1 (see Halmos, Ref. 23, sec. 8, pro- 
plem 5). 


Theorem 1,2 is due to Daniell (Refs. 3, 4) and Kolmogorov 
(Ref, 19, chapter III, sec. 4), (See also Halmos, Ref. 23, 
chapter 9, sec. 49,) Kolmogorov stated this theorem only 
for the case when(£, @)is a straight line with its usual 
topology and @=«(@); but his proof stands unchanged in the 
more general case considered here. 


Chapter 2 
Article 1. The principle of the independence of the 


"future" from the "past" for a known present was first stated 
(as applied to sequences of random variables) by Markov (Ref. 
2). This and subsequent work by Markov reveal the actual 
history of the theory of Markov random processes. Processes 
with discrete time (Markov chains) were considered almost 
exclusively in the first period of this history. 


The foundations of the general theory of Markov processes 
were laid by Kolmogorov (Ref. 18). The transition probabili- 
ties P(s, x;t,T) formed the main subject of study in Ref, 18 
and in subsequent works. The systematic investigation of 
Markov processes in the sense of random functions began some- 
what later (Levy, Doeblin, and others). The creation of a 
satisfactory, general logical concept of the Markov process 
as a random function subject to definite requirements is 
primarily due to Doob. His work on the subject is brought 
together in the monograph of Ref. 8. 


Doob started out from a general concept of probability pro- 
cess, This is defined as a function x,(w) with values in the 
measurable space (E£, 4), where the argument ¢runs over some nun- 
erical interval / and the argument » over some probability 
space (2, of, p). (It is assumed that {o: x,()€T}€of# for 
any t€/,T€@) . The condition for a process to be Markov 
is stated as: for any T€#, t<u€l, 


P (x, €T Jo} =P (x, ET]x,}  (a.c. 2, P) (1) 


(ff, denotes the c-algebra in @ generated by sets {w:x,(w)€T} 
TE sEhs<t)). 


Development of the theory of Markov processes in recent 
years has shown that: 


a) it is useful to consider events not appearing in the 
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zealgebra of, and to take as basis the strengthened variant 
of relationship (1): 


P {x ET JoM,} =P (x,€T]%}  (a.c. 2, P), 


where of, are o-algebras satisfying the inclusions % Ao#;2 
Np o&M, > A, for t>s€l > 


bd) it is advantageous to allow that the process may cut 
off at a random instant [(w). 


If we take this into account, we arrive at the general con- 
cept of Markov random function given in sec, 2.1. (This con- 
cept was first considered by Doob in Ref, 8 (and see Ref. 27). 


In the theory of Markov processes it is usually a question 
of dealing, not with a single random function, but with a 
family of such functions, corresponding to all the possible 
initial instants of time and all the possible initial states. 
It is therefore natural to assume that the theory is primarily 
concerned with what we have called Markov families of random 
functions, Various special classes of Markov family have 
been discussed in a number of works but this is the first 
time that the general case has been mentioned*), We apply 
the term "Markov process" to a somewhat simpler formation 
than the Markov family. There is no loss of generality in 
taking the process as our main subject of discussion (see 
the end of sec. 2,1) and we gain at the same time in compact- 
ness and simplicity as regards the propositions and expres- 
sions. 


Articles 2-3. Almost all the results of these articles are 
new, The basic idea of the proof of theorem 2.8 can be traced 
to Doob. In the investigation of the trajectory properties 
theorem 2.8 replaces Doob's separability theorem (Ref, Ts 
chapter 2, theorem 2,4). 


Chapter 3 


The basic ideas and results of chapter 3 are due to the 
author, The proofs of these results are published for the 
first time, Integral equation (3.76) was first deduced by 


*)The Markov random function corresponding to the transition 


function P(s, x;7,ryis quite often taken as the starting-point. 
This concept occupies an intermediate position between the 
concepts of Markov random function and Markov family. 
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another method and with different assumptions in the work by 
Blanc-Lapierre and Fortet (Ref. 1, chapter 6, sec, 14). We 
reproduce the method of deducing the equation published in 
Ref. 9. 


Chapter 4 


Theorem 4,1 can be regarded as familiar (see e.g. Ref. 7 or 
20). The basic idea of theorem 4,2 was employed, for instance, 
in Ref. 8, 


Chapter 5 

Up to 1955-1956 the strictly Markov property had undergone 
virtually no logical analysis, Many authors have used this 
property without any basis, apparently on the assumption that 
it is a general property of all Markov processes, 


Doob (Ref. 6) was the first to give a clear formation and 
proof of the strictly Markov property of certain processes, 
He showed in the work in question that all stationary Markov 
processes continuous from the right with a denumerable set of 
states are strictly Markov*). Doob's result was later exten- 
ded to wider classes of stationary Markov processes with 
denumerable sets of states (Yushkevich, Ref. 27, and Ref, 28; 
K.L, Chung, Ref, 26). 


The study of strictly Markov processes as an independent 
class was begun in 1955-1956 by Dynkin (Refs, 11, 12**), 13), 
Dynkin and Yushkevich (Ref, 16) and Ray (Ref, 225, Dynkin 
showed that, by starting from the strictly Markov property 
and attaching definite requirements to the nature of the 
trajectories, infinitesimal operators can be evaluated for 
the processes and a complete classification thus obtained of 
certain important classes of process, The work of Dynkin and 
Yushkevich (Ref. 16) gives the first general definition of 
strictly Markov processes (for the stationary case), shows 
that the class of strictly Markov processes is considerably 
narrower than the class of all Markov processes, and deduces 
the sufficient conditions for a Markov process to be strictly 
Markov. Ray's work (Ref. 22) is devoted specially to con- 
tinuous one-dimensional stationary Markov processes, The 


*)The strictly Markov property is understood in a slightly 


different sense to ours in Ref, 6, and the class of processes 
discussed there is in fact a little wider than the class of 
processes continuous from the right. 

**)Detailed proofs of the results stated in Ref, 12 are 
published in Ref. 15. 
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condition for such processes to be strictly Markov is stated 
(more weakly than in Ref. 16)*) and the necessary and suffi- 
cient conditions for the process to be strictly Markov are 
deduced in terms of the transition function**). 


Hunt's work (Ref. 24) is independent of those mentioned 
above. The strictly Markov property is stated here in a form 
suitable for stationary Markov processes with independent 
increments, and the Wiener process is shown to possess this 
property. 


The concept of strictly Markov process and the criteria for 
such a process was extended in 1957 to stationary cut-off 
processes (Blumenthal, Ref. 2) and non-stationary processes 
(Dynkin, Hef, 14; Yushkevich, Ref. 28), 


The general theory which is developed in chapter 5 includes 
the basic results of Refs, 16, 14, 28, and 2 in a modified 
form, In particular, the connexion between theorems 5.9-5.10 
and the fundamental theorem of Ref. 16 is readily seen (theo- 
rem 5 of Ref, 14 and theorem 1 of Ref, 28 occupy an inter- 
mediate position between these results). The "law of zero 
or unity" (sec, 5.21) was discovered by Blumenthal (Ref. 2), 
who also first discussed the example of sec, 5,22, 


Chapter 6 
The main sources of the theorems of this chapter are the 


results of Doeblin (Ref. 5), Dynkin (Ref. 9), Kinney (Ref. 
17) and Blumenthal (Ref. 2), In order to adapt these results 
to the definition of Markov process given in article 1 of 
chapter 2, the proofs had to be modified considerably. 


Theorem 6.2 can be looked on as a generalization of theorem 
2.1 of Ref, 2 (Blumenthal only considered the case when the 
process is given in a locally compact separable Hausdorff 
space and ¥ ={E,}is a sequence of compact sets of this 
space). Ref, 17 contains somewhat different conditions for 
boundedness, 


*)Instead of all random variables independent of the future, 
only the instants (4,4) of first departure of the trajectory 
from the interval (a2, 6‘ appear in Ray's statement. 

**)The problem of discovering the necessary and sufficient 
conditions remains open to date if the description "strictly 
Markov" is taken in the more rigid sense of Ref, 16 and the 
present book. 
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Theorem 6.3 is a "localization" of theorem V of Kinney's 
work (Ref, 17) (Kinney takes the case P= £)*), 


Theorem 6.4 is due to Doeblin (Ref. 5). 


Theorem 6,5 is due to Seregin. It is a "localization" of 
the theorem proved by Dynkin (Ref. 9) and Kinney (Ref. 17). 


Finally, theorem 6.7 is a generalization of a theorem of 
Blumenthal (Ref. 2). 


The example of a Markov, but not strictly Markov, process 
described in sec, 6.18.5 was first constructed by Yushkevich 
and published in Ref. 16, 


Addendum 
Article 1, The results described in this article are due 
to Choquet (Ref. 29). 


Article 2, The measurability properties of the instants of 
first reaching the set T (or, what amounts to the same thing, 
the instants of first departure from E\I)) were studied in 
Hunt's work (Ref. 30) for the case of non-cut-off stationary 
strictly Markov processes that have no discontinuities of the 
second kind, are continuous from the right and quasi-contin- 
uous from the left, and satisfy the condition W,,,Gc4,. 

The general theorems for wider classes of Markov process 
proved in article 2 are published for the first time, 


*)A result analogous to Kinney's theorem was obtained also 


in Ref. 9, but with a tighter condition : 
a® (8) =0(V5). 


17. 
18, 
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